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ABSTRACT 


JiThe  design  environment  is  typically  characterized  by  conflicting 
objectives  and  numerous  design  alternatives,  making  selection  of  a 
specific  system  design  problematic.  The  broad  goal  of  this  research  is 
the  development  of  a  prescriptive  decision  making  approach  which 
facilitates  the  comparison  of  a  large  number  of  design  alternatives 
with  respect  to  multiple  criteria.  The  approach  developed  considers 
both  objective  and  subjective  factors,  yet  strives  for  optimization 
rather  than  satisfying  of  the  decision  criteria.  Each  criterion  is 
considered  in  light  of  perceived  relative  weights  which  are  permitted 
to  vary  in  response  to  the  performance  achieved  for  that  objective. 
Finally,  the  decision  technique  described  provides  a  means  of  treating 
potential  interactions  among  the  multiple  criteria. 


A  design  methodology  framework  is  presented,  containing  state-of-the- 
art  features  stemming  from  the  dramatic  influence  of  systems  theory. 
These  characteristics  include  a  broad,  macro  view  of  the  system  being 
designed,  an  expansionist,  deductive  perspective  in  the  evaluation  of 
design  alternatives,  and  explicit  consideration  of  system  life  cycle 
phases.  Within  the  context  of  this  framework  methodology,  major 
decision  making  approaches  frequently  suggested  as  alternatives  for  the 
design  decision  problem  are  reviewed  and  critiqued  with  respect  to  the 
desired  features, 


The  criterion  function  approach  is  selected  as  the  most  promising 
starting  point;  however,  the  achievement  of  the  research  goal  requires 
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the  resolution  of  inconsistencies  which  occur  using  current 
implementation  methods.  The  inconsistencies  are  specified,  their 
sources  identified,  and  their  solutions  proposed.  A  demonstration  of 
the  proposed  methods  is  accomplished  using  the  design  of  an  auxiliary 
power  unit  system  for  NASA's  space  shuttle. 

The  primary  contributions  of  this  research  are  the  improved 
computational  methods  in  implementing  the  criterion  function  and  the 
concurrent  development  and  demonstration  of  a  design  decision  making 
approach  which  achieves  the  stated  desirable  characteristics. 
Potential  areas  for  future  research  include  empirical  studies 
concerning  the  use  and  success  of  design  methodologies  in  practice, 
promotion  of  the  criterion  function's  ability  to  consider  intersection 
relative  weights,  exploration  of  the  method  for  normalizing  relative 
weights,  and  implementation  of  the  criterion  function  in  NASA  design 
decision  making. 
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Chapter  1 


COMPLEX  SYSTEM  DESIGN 


System  Design/Planning 

Throughout  history,  man's  success  has  depended  on  his  ability  to 
master  his  environment.  In  some  cases,  physical  strength  and  agility 
have  been  adequate  to  achieve  this  mastery.  However,  we  generally 
attribute  the  majority  of  man's  success  to  his  unique  mental 
capabilities.  In  particular,  we  consider  man  superior  to  his  animal 
peers  primarily  in  terms  of  his  ability  to  understand  and  reason. 
Even  in  basic  matters  of  survival,  man  frequently  depends  on  his 
ability  to  analyze  the  issues  relevant  to  a  given  situation  and  then 
synthesize  a  plan  of  action  which  will  achieve  some  desired 
objective (s) . 

In  the  most  elementary  cases,  activities  may  be  planned  and 
carried  out  in  a  simple,  informal  manner.  However,  as  history  has 
progressed,  man's  objectives  have  become  increasingly  sophisticated 
and  more  complex.  Large  undertakings  such  as  governmental 
organization  and  technological  innovation  have  required  that  many 
people,  tools,  and  resources  be  integrated  to  achieve  a  unified  goal. 
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Consequently,  the  effort  and  complexity  of  the  planning  activities 
have  increased  in  a  corresponding  manner.  For  example,  the  building 
of  the  pyramids  and  the  development  of  the  Roman  empire  required  far 
more  intricate  planning  activities  than  anything  which  had  gone 
before.  Similarly,  modern  activities  such  as  space  programs  and 
social  welfare  systems  tax  man's  ability  to  reason  far  beyond  what  was 
previously  possible.  Therefore,  as  time  has  passed,  the  need  for  more 
effective,  formal  approaches  to  design  and  planning  has  continued  to 
increase  and  can  be  expected  to  do  so  for  the  foreseeable  future . 


Perhaps  more  than  ever,  effective  system  design  is  perceived  as 
key  to  the  resolution  of  today's  goals  and  problems: 


Systems  methodology  and  design  (SMD)  is  emerging  as  a 
concept  of  and  reasoning  principles  for  integrating 
traditional  engineering  problem  solving  with  systems  theory, 
management  science,  behavioral  decision  theory,  and  planning 
and  design  approaches.  The  resultant  combination  is  a 
methodological  and  creative  concept  for  attacking  and 
solving,  with  a  long  range  perspective,  today's  problems  in 
organizations  in  general  and  in  engineering  and  industry  in 
particular.  At  a  time  when  poor  growth  in  productivity  and 
management  performance  are  being  shown  to  be  important 
obstacles  to  healthy  economic  growth,  SMD  might  be  an 
important  concept  for  both  short-  and  long-term  recovery. 
(Nadler ,  1985) 


This  research  is  another  step  in  the  on-going  effort  to  provide 
the  planning  methods  and  tools  necessary  to  perpetuate  the  achievement 
of  man's  goals.  Since  it  is  unreasonable  to  assume  that  all  of 
mankind's  goals  have  been  explicitly  defined,  any  methods  developed  as 
part  of  this  research  must  be  conceived  and  described  in  the  broadest 


maimer  possible  in  order  to  achieve  flexibility  of  application. 
However,  the  potential  ambiguity  of  discussing  broad  concepts  suggests 
that  we  precede  further  discussion  with  the  following  definitions. 

Definition  of  a  System 

Throughout  this  research,  we  will  refer  to  the  design  or  planning 
of  systems  without  necessarily  defining  what  those  systems  may  be. 
The  definition  we  choose  to  use  for  the  term  "system"  is: 

Any  set  of  objects  and/or  attributes  and  the  relationships 

among  them.  (Ostrofsky,  1988) 

This  definition  is  intentionally  obtuse.  A  system  may  be  a 
physical  object  or  collection  of  objects.  By  contrast,  it  may  be  no 
more  than  a  set  of  operating  procedures  or  rules .  A  system  may 
include  people,  computer  programs,  or  any  combination  of  these 
possibilities.  In  fact,  most  large  scale  systems  will  normally 
include  some  combination  of  all. 

We  closely  follow  the  orientation  of  systems  theory  in  which  it 
is  generally  assumed  that  a  system  exists  for  some  purpose(s). 
Following  this  line  of  reasoning,  a  system  is  typically  considered  to 
involve  some  combination  of  inputs,  outputs,  and  a  transformation 
process  which  converts  inputs  to  outputs .  Although  many 
representations  are  possible,  we  will  generally  refer  to  the  systems 
model  used  by  Asimow  (1962)  in  which  a  system  is  described  in  terms  of 


its  environmental  inputs,  intended  inputs,  desired  outputs,  and 
undesired  outputs  (Figure  1) . 


4 


Definition  of  Design 

In  order  to  achieve  a  particular  system,  a  series  of  purposeful 
activities  must  be  defined  and  undertaken.  We  refer  to  this  set  of 
activities  as  system  design.  While  the  term  "design"  typically 
generates  a  mental  image  of  some  physical  object  or  system,  it  must  be 
remembered  that  our  definition  of  a  system  is  not  so  restrictive. 
Consequently,  we  consider  the  more  generic  term  "planning"  to  be 
synonymous  with  design.  The  two  terms  will  be  used  interchangeably  in 
this  research . 


As  the  art  and  science  of  design  have  progressed  through  history, 
many  developments  have  emerged  which  have  left  a  lasting  influence  on 
current  approaches  (Asimow,  1962;  Buhl,  1961;  de  Neufville  and  Marks, 
1974;  Department  of  Defense,  1968;  Dixon,  1966;  Gosling,  1962; 
Gregory,  1966;  Hall,  1962,  1969;  Haupt,  1978;  Hill,  1970;  Holzman, 
1979;  Jones,  1970;  Kline;  Meredith,  1973;  Middendorf,  1969;  Ostrofsky, 
1977d;  Sage,  1977,  1986;  Simon,  1975;  U.S.  Air  Force,  1966,  1974; 
Vidosic,  1969;  Wallace,  1978;  Wilson,  1965;  Woodson,  1966;  Wymore, 
1976) .  Although  these  developments  are  too  numerous  to  list 
exhaustively  or  describe  individually,  the  following  discussion 


highlights  some  of  the  significant  issues  emerging  from  the  intense 
research  devoted  to  this  area  in  recent  decades . 
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Design  Phases 

Given  our  description  of  design  as  a  series  of  purposeful 
activities,  it  is  logical  that  many  researchers  and  authors  have 
focused  on  defining  the  generic  logical  sequence  and  grouping  of 
required  activities.  These  groupings  are  typically  referred  to  as 
design  steps  or  phases. 

Some  of  the  more  prominent  delineations  of  design  phases  are 
shown  in  Figure  2.  Two  of  the  more  influential  schemes  were  those  of 
Hall  (1962)  and  Asimow  (1962).  As  is  the  case  with  most  such 
classification  schemes,  the  earliest  design  stages  are  concerned  with 
the  broadest  issues  of  the  effort  while  the  later  stages  serve  to 
specify  increasingly  finer  levels  of  detail.  For  example,  stages  such 
as  program  planning  and  feasibility  study  generally  begin  with  a 
broad,  yet  formal  statement  of  the  problem  and  description  of  the 
relevant  influences  and  participants  in  the  design  environment. 
Subsequent  stages  strive  first  for  determination  of  a  general  solution 
approach  and  then  for  a  detailed  description  of  how  that  approach  is 
to  be  implemented. 


Similar  in  concept  to  those  proposed  by  Hall  and  Asimow,  the 


Hall  (1962) 
Systems  Engineering 
Phases 

Program  Planning 
Project  Planning 
System  Development 
Studies  During 
Development 
Current  Engineering 


Asimow  (1962) 
Design  Phases 


DoD  (1968) 
Design  Phases 


Feasibility  Study 
Preliminary  Design 
Detail  Design 
Planning  for  Production 
Planning  for  Distribution 
Planning  for  Consumption/ 
Operations 

Planning  for  Retirement 


Formulation 

Contract 

Definition 

Development 

Production 

Operational 


Hall  (1969) 
Systems  Engineering 
Phases 

Program  Planning 

Project  Planning 

System  Development 

Production 

Distribution 

Operation 

Retirement 


Wymore  (1976) 
Design  Steps 

-  Problem  Definition 

-  Preliminary  Design 

-  Final  Design 

-  Implementation 

-  Acceptance 

-  Testing 

-  Operation 

-  Retirement 


Sage  (1977) 

Design  Phases 

-  Program  Planning 

-  Proj  ect  Planning 

-  System  Development 

-  Production 

-  Distribution 

-  Operation 

-  Retirement 


Ostrofsky  (1977d) 
Design  Phases 

-  Feasibility  Study 

-  Preliminary  Design 

-  Detail  Design 

-  Production 

-  Distribution 

-  Consumption/Operations 

-  Retirement 


Figure  2.  Design  Phases 
(Adapted  from  Folkeson,  1982) 
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design  phases  specified  by  the  Department  of  Defense  (1968)  were  also 
chosen  to  highlight  logical  review  points  between  phases,  facilitating 
the  monitoring  of  contracted  design  efforts  common  to  Air  Force  system 
development . 

Subsequent  design  phase  descriptions  (Woodson,  1966;  Hall,  1969; 
Wymore,  1976;  Sage,  1977;  Ostrofsky,  1977d)  were  largely  refinements 
of  these  earlier  schemes.  Although  the  phase  names  and  the  grouping 
of  individual  tasks  and  activities  vary  slightly  from  scheme  to 
scheme,  the  overall  content  is  quite  similar.  The  general  conclusion 
to  be  drawn  from  comparing  the  alternative  descriptions  is  that  the 
exact  sequence  and  method  of  implementation  is  less  important  than  the 
need  to  insure  that  the  commonly  agreed  on  tasks  are  accomplished 
(Wallace,  1988). 


System  Life  Cycle  Phases 

Early  attempts  to  define  the  design  phases  generally  focused  on 
visualizing  the  target  system  in  steady  state  operation.  The  primary 
goal  was  perceived  as  maximizing  the  potential  performance  of  the 
system  during  the  operational  portion  of  its  existence.  Gradually, 
experience  led  many  designers  to  begin  to  recognize  the  importance  of 
considering  other  phases  of  the  system's  life  when  first  specifying 
what  characteristics  the  system  should  possess.  In  particular,  it 
became  apparent  that  other  characteristics  of  importance  to  the 
ultimate  success  of  the  system  had  to  be  considered  from  the  outset  of 


the  design  effort  and  built  in.  Such  characteristics  might  include 
the  system's  producability ,  ease  of  implementation/installation, 
reliability,  recyclability,  disposability,  etc.  Since  these 
characteristics  typically  have  a  substantial  impact  on  the  requirement 
for  and  use  of  equipment,  people,  or  other  system  resources  they 
cannot  be  effectively  or  efficiently  added  on  after-the-fact  to  an 
existing  system. 

As  a  result  of  these  realizations,  researchers  sought  to  describe 
general  phases  in  the  life  cycle  of  any  intended  system  so  that  these 
phases  could  become  the  targets  of  specific  planning  activities  during 
the  design  process  (Asimow,  1962;  Department  of  Defense,  1968; 
Ostrofsky,  1977d;  Sage,  1977;  U.S.  Air  Force,  1966,  1974;  Wymore, 
1976).  Note,  for  example,  that  Asimow's  (1962)  last  four  design 
phases  (Figure  2)  expressly  addressed  what  he  perceived  to  be  the  four 
primary  life  cycle  phases:  production,  distribution, 
operations/consumption,  and  retirement.  Production  refers  to  the 
initial  creation  of  the  system.  Distribution  refers  to  all  activities 
necessary  to  deploy  and/or  implement  the  system  in  its  intended 
environment.  Operations  refers  to  the  performance  of  the  primary 
functions  for  which  the  system  was  originally  intended.  Note  that  the 
possible  substitution  of  the  term  "consumption"  supports  the  broad 
definition  of  a  system  we  have  chosen  to  use:  some  systems  are 
consumed  rather  than  operated.  The  retirement  phase  of  the  life  cycle 
is  perhaps  the  most  frequently  neglected  in  terms  of  design.  It  is 
not  surprising  that  many  designers  fail  to  consider  the  potential 
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consequences  of  the  ultimate  obsolescence  of  their  system.  While 
phased  revision  or  perpetual  modification  is  certainly  a  feasible 
alternative  for  some  systems,  others  will  require  consideration  of 
disposal  or  neutralization  activities.  Increased  awareness  in  recent 
years  of  environmental  consequences  has  highlighted  the  success  or 
failure  of  various  system  designs  in  this  phase.  Characteristics  such 
as  biodegradability  and  recyclability  have  begun  to  assume 
significantly  more  attention  in  modem  design  efforts. 

As  with  delineation  of  the  design  phases,  the  system  life  cycle 
phases  could  easily  be  grouped  or  described  differently  without 
degradation  as  long  as  the  content  of  the  issues  considered  remained 
relatively  the  same.  However,  it  is  interesting  to  note  that  later 
design  schemes  (Hall,  1969;  Wymore,  1976;  Sage,  1977)  have  retained 
the  four  stage  life  cycle  described  above,  largely  unchanged. 

Influence  of  the  Systems  Approach 

Boulding  (1956)  described  the  quest  of  General  Systems  Theory  as 
the  development  of  "a  body  of  systematic  theoretical  constructs  which 
will  discuss  the  general  relationships  of  the  empirical  world."  More 
specifically,  Johnson,  Kast,  and  Rosenzweig  (1964)  stated  that  "the 
aim  of  systems  theory  for  business  is  to  develop  an  objective, 
understandable  environment  for  decision  making." 


The  introduction  and  subsequent  popularity  of  systems  theory  and 
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the  broad  "systems"  perspective  have  had  significant  and  lasting 
effects  on  the  theory  and  practice  of  management  (Checkland,  1981; 
Cleland  and  King,  1972;  Forrester,  1968;  Johnson,  Kast,  and 
Rosenzweig,  1970;  Van  Gigch,  1978).  The  following  paragraphs  briefly 
summarize  the  ways  in  which  these  influences  have  carried  over  into 
the  arena  of  design. 

Macro  vs.  Micro  Level  Design 

Even  today,  the  topic  of  design  tends  to  evoke  images  of  detailed 
blueprints  or  wiring  diagrams  for  a  specific  tangible  product.  The 
design  is  frequently  perceived  to  be  confined  to  a  description  of  the 
characteristics  to  be  embodied  within  the  physical  confines  of  that 
product.  However,  one  of  the  great  influences  of  the  systems  movement 
was  the  concept  of  defining,  a  priori,  a  system's  boundaries  as  well 
as  its  interfaces  with  its  environment.  This  seemingly  simple  concept 
highlighted  the  realization  that  a  system  may  be  defined  at  virtually 
any  level.  In  addition,  the  specification  of  interfaces  requires  the 
recognition  that  any  system  is,  itself,  contained  within  another, 
larger  system. 

These  realizations  encouraged  designers  to  define  an  appropriate 
level  of  focus  for  their  design  activities  and  to  consider  the 
interactions  between  system  and  environment.  The  natural  consequence 
of  these  influences  has  frequently  resulted  in  a  much  broader  level  of 
system  definition  than  had  previously  been  the  case.  This  broader, 
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macro  level  perspective  has  facilitated  the  ability  of  the  designer  to 
ensure,  for  example,  that  not  only  is  the  tangible  product  accounted 
for,  but  also  the  required  support  subsystems  necessary  for  the 
sustained  success  of  the  system  in  its  intended  operating  environment. 
Such  considerations  have  complemented  the  broadened  awareness  of  the 
system  life  cycle  phases. 

Another  offshoot  of  the  broadened  systems  perspective  has  been  an 
increased  willingness  to  apply  structured  design  methods  to  much 
larger,  more  interdisciplinary  problems  than  were  previously 
considered  feasible  (de  Neufville  and  Keeney,  1974;  de  Neufville  and 
Marks,  1974;  Department  of  Defense,  1968;  Sage,  1977;  U.S.  Air  Force, 
1966,  1974;  Wymore,  1976).  Researchers  recognized,  for  example,  that 
the  same  methodology  used  to  design  a  vacuum  cleaner  might  lend 
support  to  the  effective  planning  of  an  airport  or  a  medical  service 
system.  It  is  this  influence  which  has  led  to  the  broad  definition  of 
system  design  we  provided  above. 

Reductionism  vs.  Expansionism 

Prior  to  the  advent  of  systems  theory,  it  was  common  practice  to 
undertake  the  design  of  a  system  by  first  decomposing  the  system  into 
its  constituent  subsystems  and  then  designing  or  optimizing  each 
subsystem  (Nadler,  1985).  The  final  system  was  then  synthesized  by 
combining  the  optimized  subsystems.  Implicitly,  each  subsystem  was 


considered  independent  of  the  others.  This  perspective  is  referred  to 
as  "reductionism"  (Ackoff,  1979a). 

However,  Sage  (1981:  76)  observes  that  expert  planning  is 
characterized  by  a  wholistic  approach  rather  than  a  decomposed  one. 
Closely  related  to  the  definition  of  system  boundaries  and  the  macro 
level  perspective  of  design  is  the  focus  of  systems  theory  on  the 
performance  of  the  integrated  system  within  the  context  of  its 
surrounding  system  environment.  Rather  than  suboptimizing,  the 
systems  approach  encourages  consideration  of  the  interaction  among 
subsystems  and  is,  therefore,  concerned  with  the  performance  of  the 
system  as  a  whole  (Ackoff,  1979b).  This  approach  is  referred  to  as 
"expansionism"  (Ackoff,  1979a;  Nadler,  1985).  Subsystems  are  viewed 
in  a  synergistic  sense  with  the  whole  being  greater  than  the  sum  of 
its  parts. 

Inductive  vs.  Deductive  Design 

Another  characteristic  of  early  design  methods  was  the  reliance 
on  creativity  to  visualize  a  specific  desireable  system  alternative 
early  in  the  design  process.  This  envisioned  system  then  served  as 
the  basis  for  an  inductive  approach  (Nadler,  1985)  to  its  own  design. 
Sometimes  referred  to  as  "point  design",  this  approach  implicitly 
assumed  that  the  point  envisioned  by  the  designer  was  the  best 
alternative  to  achieve  the  needs  of  the  system.  Therefore,  the 
balance  of  the  design  activities  focused  on  the  ability  of  the 


designer  to  achieve  that  point.  Simply  put,  this  approach  starts  with 
a  specific  solution  and  then  endeavors  to  make  that  solution  fit  the 
problem. 

By  contrast,  the  systems  perspective  encourages  a  deductive 
approach  to  the  design  effort  (Nadler,  1985).  This  method  focuses  on 
the  situational  needs  and  then  explores  broader  theoretical 
alternatives  in  a  search  for  that  specific  implementation  which  best 
satisfies  the  needs.  Ackoff  (1979b)  refers  to  this  approach  as 
"idealized  design"  in  which  the  desired  system  is  actually  a  response 
to  the  perceived  needs  and  the  specific  design  emerges  as  a  means  to 
that  end. 


Iterative  Nature  of  Design 

A  final  noteworthy  development  in  design  methodology  has  been  the 

increased  recognition  of  the  iterative  nature  of  the  design  process. 

As  mentioned  earlier,  the  purposeful  activities  of  design  are 

generally  perceived  as  a  sequence  of  decisions  or  specifications  which 

proceed  to  the  ultimate  solution  at  increasing  levels  of  detail. 

However,  as  the  designer  proceeds  through  this  series  of  decisions  he 

gains  additional  information  and  insight  beyond  that  which  he 

possessed  earlier  in  the  process.  In  some  cases,  this  new  information 

may  suggest  reconsideration  of  decisions  made  earlier  in  the  process 

(Ostrofsky,  1977d) .  Ackoff  (1979b)  asserts  that: 

.  .  .  plans  should  be  continuously  revised  in  light  of  (i) 
their  performance,  (ii)  unexpected  problems  and 


opportunities  that  arise,  and  (iii)  the  latest  information, 
knowledge  and  understanding  available,  much  of  which  is 
derived  from  the  implementation  process. 

Consequently,  the  ultimate  quality  of  the  system  design  is  enhanced  if 

the  designer  returns  to  the  earlier  point  in  the  process  and 

reiterates  through  the  intervening  steps. 

The  reiteration  process  may  be  perceived  as  costly  from  a  short¬ 
sighted  point  of  view  due  to  its  impact  on  the  design  project 
schedule.  However,  it  is  frequently  the  case  that  both  the  quality  of 
the  final  system  and  the  speed  with  which  it  is  successfully  realized 
are  improved  when  iteration  is  accomplished.  Consequently,  productive 
iteration  is  encouraged  rather  than  discouraged.  In  fact,  recent 
researchers  in  the  design  of  information  systems  have  encouraged 
evolutionary  approaches  in  which  the  objective  is  to  obtain  a  working 
system  at  the  earliest  possible  point  and  then  focus  on  the  process  of 
adjusting  the  system  to  maximize  its  performance  based  on  direct 
observation  of  its  success  or  failure  within  the  actual  operating 
environment  (Keen,  1980;  Alavi  and  Henderson,  1981).  A  corollary 
principle  espoused  by  Asimow  (1962)  is  the  "Principle  of  Least 
Commitment".  This  principle  states  that  no  irreversible  decision 
should  be  made  until  it  absolutely  must  be  made. 

Design  Decision  Making 


We  have  characterized  the  design  process  as  consisting  of  a 
series  of  decisions  which  must  be  made  (either  explicitly  or 
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implicitly)  by  the  designer.  Starr  (1973)  has  suggested  that  such 
broad  design  decision  making  problems  can  be  visualized  as  being 
composed  of  three  problems:  the  search  problem,  the  creation  (or 
invention)  problem,  and  the  decision  problem.  Foremost  among  these 
decisions  is  the  determination  of  which  design  alternative  should  be 
pursued  to  implementation:  the  decision  problem.  Regardless  of  the 
design  approach  used,  this  decision  is  central  to  the  ultimate  success 
or  failure  of  the  system.  Nadler  (1967)  has  identified  the  decision 
making  process  used  to  produce  the  target  system  as  one  of  the  primary 
research  issues  in  the  area  of  system  design  methodology.  It  is  this 
decision  making  process  which  serves  as  the  focus  of  this  research. 
Therefore,  we  will  subsequently  refer  to  this  process  simply  as  design 
decision  making.  In  the  following  paragraphs,  we  highlight  the 
characteristics  of  the  design  decision  process. 

Complex  Decision  Environment 

The  success  or  quality  of  design  alternatives  in  most  practical 
environments  (particularly  for  large-scale  systems)  typically  involves 
consideration  of  multiple  objectives  (Easton,  1973a).  Unfortunately, 
it  is  common  for  the  optimization  of  some  characteristics  to  run 
counter  to  the  realization  of  other  system  design  objectives  (e.g. 
cost  vs.  schedule).  Consequently,  the  design  environment  is  typically 
characterized  by  the  need  to  simultaneously  satisfy  multiple 
conflicting  objectives. 
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It  is  also  true  that  today's  design  environment  is  characterized 
by  a  high  degree  of  complexity.  Rapid  advances  in  computing  and 
communications  have  permitted  the  inclusion  of  sophisticated  feedback 
and  control  systems  in  even  the  simplest  of  modern  products  and 
systems .  This  has  meant  not  only  that  greater  technical  complexity  is 
present  in  design  alternatives,  but  also  that  the  number  of 
alternatives  available  in  the  design/planning  process  has  increased 
exponentially  with  the  possible  combinations  of  hardware,  software, 
and  procedures  (Easton,  1973b;  Starr  and  Greenwood,  1977). 

The  planning  and  psychology  literature  clearly  substantiate  the 
difficulty  of  the  human  mind  in  dealing  effectively  and  consistently 
with  such  complexity,  conflicting  objectives,  and  large  numbers  of 
alternatives  (Bowman,  1963;  Brunswik,  1947;  Dawes,  1971;  Dawes  and 
Corrigan,  1974;  Ebert  and  Kruse,  1978;  Goldberg,  1970,  1976;  Hammond, 
1955;  Hamner  and  Carter,  1975;  Hogarth  and  Makridakis,  1981a,  1981b; 
Hurst  and  McNamara,  1967;  Johnson  and  Payne,  1985;  Kunreuther,  1969; 
Todd,  1954;  Tversky  and  Kahneman,  1974).  Consequently,  there  is  a 
continuing  need  for  usable,  yet  objective  decision  aids  in  the 
design/planning  process. 

Prescriptive  vs.  Descriptive  Decision  Models 

Decision  making  models  are  typically  either  descriptive  or 
prescriptive  in  nature.  Descriptive  models  attempt  to  describe  or 
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explain  how  decisions  are  currently  made.  Prescriptive  models,  on 
the  other  hand,  attempt  to  explain  how  decisions  should  be  made. 

Descriptive  decision  models  are  predominant  in  fields  such  as 
psychology  and  management  information  systems.  The  objective  in  these 
areas  is  to  understand  not  just  what  decisions  are  made  in  a  given 
situation,  but  also  to  peer  inside  the  "black  box"  of  the  decision 
process  to  understand  why  a  particular  decision  is  reached  (Todd  and 
Benbasat,  1987).  The  rationale  for  these  models  lies  in  the  belief 
that  if  we  understand  the  strengths  and  weaknesses  of  the  human 
decision  process  we  may  be  able  to  accentuate  the  strengths  and 
counter  the  weaknesses  in  such  a  way  that  the  quality  of  the  resulting 
decisions  is  enhanced.  These  models  frequently  include  consideration 
of  factors  such  as  cognitive  style  and  communication  patterns  in 
addition  to  raw  problem  data. 

By  contrast,  prescriptive  decision  models  rarely  attempt  to 
mirror  the  actual  human  decision  process.  Instead,  the  objective  of 
these  models  is  to  be  able  to  take  the  same  input  pieces  of 
information  and  produce  an  output  decision  which  is  better  than  that 
which  would  have  been  achieved  using  other  methods.  The  focus  is  on 
the  quality  of  results,  not  on  the  method  of  achieving  those  results. 
Prescriptive  models  are  typically  quantitative  in  nature.  It  is  this 
form  of  decision  making  model  which  will  be  pursued  in  this  research. 
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Subjective  and  Objective  Considerations 


Most  complex  decisions  involve  both  subjective  and  objective 
considerations.  Structured  design  methodologies  and  prescriptive 
decision  models  generally  focus  on  objective  factors  almost  to  the 
exclusion  of  subjective  ones.  Yet  in  the  absence  of  such  structure, 
we  typically  observe  that  humans  give  subjective  factors  very  high 
priority.  This  may  be  partially  explained  by  the  perceived  difficulty 
of  making  objective  assessments.  However,  another  explanation  may  lie 
in  the  recognition  that  subjective  factors,  while  perhaps  not  as 
amenable  to  quantification,  are  nonetheless  perceived  as  crucial 
ingredients  in  the  decision  making  process.  Ackoff  (1979a) 
persuasively  argues  the  narrow-mindedness  and  ultimate  futility  of 
attempting  to  restrict  our  decision  making  considerations  to  solely 
objective  judgments: 

Objectivity  is  not  the  absence  of  value  judgments  in 
purposeful  behaviour.  It  is  the  social  product  of  an  open 
interaction  of  a  wide  variety  of  subjective  value 
j udgments .  Objectivity  is  a  systematic  property  of  science 
taken  as  a  whole,  not  a  property  of  individual  researchers 
or  research.  It  is  obtained  only  when  all  possible  values 
have  been  taken  into  account;  hence,  like  certainty,  it  is 
an  ideal  that  science  can  continually  approach  but  never 
attain.  That  which  is  true  works  and  it  works  whatever  the 
values  of  those  who  put  it  to  work.  It  is  value  -  full .  not 
value -free . 

Therefore,  the  ideal  decision  model  should  provide  the  structure  to 
consistently  include  both  objective  and  subjective  considerations 
(Hall,  1962). 
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Relative  Importance  of  Criteria 


We  have  previously  described  the  design  decision  environment  as 
being  characterized  by  multiple  (typically  conflicting)  objectives. 
However,  it  is  possible  and,  in  fact,  quite  likely  that  not  all 
decision  criteria  will  be  equal  in  importance.  Therefore,  one 
challenge  to  be  met  by  decision  models  is  the  need  to  treat  multiple 
criteria  of  unequal  importance.  This  weighting  process  may  well  be 
one  of  the  ways  in  which  subjective  considerations  enter  the  decision 
process . 


An  additional  complication  in  the  treatment  of  relative  weights 

is  the  recognition  that  the  relative  importance  of  criteria  may  vary 

as  the  performance  of  a  system  varies.  In  the  context  of  clinician 

judgement,  Hoffman  (1960)  has  observed  that: 

.  .  .  clinicians  seldom  believe  that  linear  functions  best 
describe  relationships  between  information  and 
characteristics  being  judged.  It  may,  in  fact,  be  closer  to 
the  truth  that,  at  least  for  some  classes  of  information, 
extreme  scores  are  more  decisive  in  judgement  than  are 
scores  in  the  middle  range  .  .  .  For  some  (clinicians) ,  scores 
above  or  below  some  arbitrary  value  may  carry  no  added 
significance  whatsoever. 

More  directly  related  to  the  context  of  design,  Shestakov  (1983) 
identified  several  examples  where  the  decision  maker's  preferences  may 
change  when  performance  measures  cross  some  threshold  value,  i.e. 
preferences  may  vary  within  intervals. 


As  the  number  of  preference  intervals  approaches  infinity  or  the 
ability  to  identify  discrete  interval  breaks  diminishes ,  the  concept 


of  varying  relative  weights  may  be  seen  to  evolve  toward  approximation 
by  continuous,  perhaps  even  nonlinear,  functions.  In  the  business  or 
management  context,  cost  considerations  may  take  on  increasing 
importance  relative  to  other  design  criteria  as  the  cost  of  a 
particular  alternative  approaches  budgetary  constraints.  Conversely, 
while  some  minimum  level  of  safety  may  be  considered  crucially 
important,  additional  levels  of  safety  may  be  perceived  as  yielding 
diminishing  returns . 


Satisficing  vs.  Optimization 

From  a  descriptive  perspective,  Simon  (1955;  1958)  has  suggested 
that  the  decision  process  typically  involves  the  evaluation  of 
alternatives  with  respect  to  some  minimum  standard  of  performance. 
Decision  making  is  characterized  as  involving  a  search  for 
alternatives  which  meet  this  minimum  desired  level  of  performance: 
solutions  which  are  "good  enough".  This  concept  is  referred  to  as 
satisficing. 

By  contrast,  prescriptive  decision  models  typically  attempt  to 

\y*‘ 

identify  the  alternative  whose  performance  is  expected  to  be  the  best 
possible  with  respect  to  the  design  criteria.  This  goal  is  referred 
to  as  optimization. 

Folkeson  (1982)  suggests  that  satisficing  may  be  perceived  as  a 
practical  form  of  optimization  in  which  rational  trade  offs  are  made 
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between  the  complexity  of  the  decision  process  and  the  importance  of 
the  potential  outcome.  The  more  valuable  the  desired  outcome,  the 
greater  the  motivation  to  seek  optimization.  While  this  idea  seems 
perfectly  valid,  it  still  infers  that  large-scale  design  efforts  with 
far-reaching  consequences  merit  the  pursuit  of  optimality. 

However,  given  the  complications  of  multiple  conflicting 
objectives  and  the  presence  of  subjective  considerations,  it  is 
clearly  debateable  whether  provable  optimality  is  a  realistic 
objective.  Therefore,  Keen  (1977)  has  observed  that  optimality  in 
such  situations  is  an  evolving  concept  which  eludes  absolute 
definition,  depending  instead  on  the  context  of  the  problem  and  the 
needs  and  ability  of  the  decision  maker.  Consequently,  we  are  forced 
to  base  our  definition  of  optimality  on  selection  of  an  appropriate 
theory  and  subsequent  faithful  implementation  of  that  theory.  Using 
such  an  approach,  we  hope  to  emerge  with  a  decision  which  was 
logically  derived  and  is  reasonably  defensible  within  the  context  of 
our  problem,  despite  our  inability  to  assert  that  it  is  indisputably 
and  unchangeably  the  "ideal"  solution. 

Interaction  Among  Criteria 

As  discussed  earlier,  design  activities  prior  to  the  advent  of 
systems  thinking  were  frequently  undertaken  using  a  reductionist 
perspective  in  which  subsystems  were  optimized  under  the  assumption  of 
independence.  While  the  expansionist  perspective  of  systems  theory 
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has  helped  combat  this  problem  in  terms  of  the  relationships  among  the 

subsystems  being  designed,  a  parallel  problem  may  be  seen  to  exist 

with  respect  to  the  multiple  design  criteria.  Specifically,  it  is 

quite  reasonable  to  expect  that  factors  which  influence  the 

performance  of  a  design  alternative  on  one  criterion  may  also  affect 

the  performance  of  that  alternative  on  some  other  criterion. 

Therefore,  any  decision  making  approach  which  treats  design  criteria 

as  independent  may  fail  to  recognize  the  effect  of  interactions  among 

criteria,  thus  increasing  the  risk  of  an  improper  optimal  choice. 

Consequently,  Hoffman  (1960)  advises  that: 

The  inclusion  of  interaction  terms  in  a  (decision)  model 
takes  account  of  the  possibility  that  for  a  particular  judge 
the  interpretation  of  one  item  of  information  may  be 
contingent  upon  a  second  ...  it  is  possible  to  include  terms 
involving  higher  order  interactions  or  other  postulated 
functional  relationships  among  the  information  variables. 

Objectives  of  This  Research 

The  issues  discussed  in  the  preceding  paragraphs  support  Nadler's 
(1967)  promotion  of  the  need  for  continued  research  in  design  decision 
making.  This  need  has  been  repeated  more  recently  (Rabins  et  al, 
1986;  Tompkins,  1989)  and  is  referred  to  by  Starr  (1973)  as  simply 
"the  decision  problem".  Using  these  issues  as  a  guideline,  we  define 
the  objective  of  this  research  to  be  the  development  of  a  prescriptive 
design  decision  making  approach  which: 

1)  supports  comparative  evaluation  of  a  large  number  of  design 


alternatives , 


2)  considers  multiple,  potentially  conflicting,  objectives, 

3)  considers  both  objective  and  subjective  relevant  factors, 

4)  permits  relative  weights  which  vary  both  among  criteria  and 
within  the  range  of  possible  criteria  values, 

5)  seeks  decisions  which,  within  the  context  of  the  defined 
problem,  are  optimal  in  the  sense  suggested  by  Keen  (1977)  and 
discussed  earlier  in  this  chapter,  and 

6)  considers  interactions  among  criteria. 

In  achieving  this  objective,  the  research  will  contribute  a 
method  which  responds  to  decision  making  needs  in  the  design  of  new, 
increasingly  complex  systems.  In  addition,  the  method  developed  will 
provide  a  significantly  more  realistic  and  effective  means  for 
resolving  previously  defined  problems. 


Organizatior 


Chapter  2  facilitates  examination  of  the  decision  process  by 
describing  a  specific  formal  design  methodology  which  provides  a 
suitable  framework  for  design  decision  making.  Chapter  3  contains  a 
review  of  significant  existing  prescriptive  approaches  to  the  decision 
making  process,  an  examination  of  their  respective  and  collective 
strengths  and  weaknesses  in  the  design  context,  and  the  selection  of 
the  method  which  serves  as  the  basis  for  development  of  the  desired 
approach.  Chapter  4  provides  the  theoretical  development  of  the 
desired  approach.  Chapter  5  contains  a  demonstration  of  the  suggested 
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approach  via  application  to  an  actual  problem.  Finally,  Chapter  6 
summarizes ,  including  a  review  of  the  achievement  of  the  research 
objectives,  an  assessment  of  the  limitations  of  the  research,  and 
suggested  important  areas  for  future  research. 


Chapter  2 


DESIGN/PLANNING  METHODOLOGY 

Johnson,  Kast,  and  Rosenzweig  (1964)  have  observed  that  "if  the 
system  within  which  managers  make  the  decisions  can  be  provided  as  an 
explicit  framework,  then  such  decision  making  should  be  easier  to 
handle."  Therefore,  to  facilitate  the  development  of  the  decision 
making  process  which  satisfies  the  stated  research  objective,  it  is 
first  necessary  to  establish  the  context  of  a  design  methodology  which 
implements  the  significant  methodological  developments  described 
earlier.  It  can  be  argued  that  many  approaches  meet  this  requirement. 
While  the  names  and  sequence  of  steps  differ  from  one  approach  to 
another,  most  contain  only  slight  variations  of  the  same  basic 
principles.  Since  the  focus  of  this  research  is  on  the  decision 
making  process,  it  is  sufficient  for  our  purposes  simply  to  identify  a 
background  methodology  which  possesses  the  desired  characteristics 
rather  than  attempting  to  identify  the  single  best  methodology. 
Therefore,  this  chapter  describes  such  a  methodology  and  demonstrates 
its  embodiment  of  the  desired  features  without  asserting  that  it  is 
the  only  such  approach. 

The  design  morphology  formulated  by  Asimow  (1962)  was  based 
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simply  on  his  observations  of  a  sequence  of  decisions  which  must  be 
made  (either  explicitly  or  implicitly)  in  any  design/planning  process. 
Ostrofsky  (1977d)  later  developed  and  extended  Asimow's  design 
morphology  to  its  present  form  (Figure  3). 

The  stages  in  Figure  3  are  broadly  grouped  into  two 
classifications  based  on  the  system  life  cycle  concept:  the 
Design/Planning  phases  and  the  Production/Consumption  phases.  The 
former  include  those  phases  which  span  the  activities  from  the 
conceptualization  of  the  system  until  its  implementation  and 
constitute  the  design  methodology  itself.  The  last  four  phases 
represent  the  system  life  cycle  subsequent  to  the  design  activities 
and  represent  the  target  activities  of  the  design  effort.  Each  of  the 
steps  included  in  the  Design/Planning  phases  must  be  accomplished, 
largely  in  the  sequence  shown.  This  is  not  to  say  that  each  step  is 
final.  On  the  contrary,  the  discovery  of  additional  information  or 
the  determination  that  previous  steps  were  inadequately  accomplished 
is  intended  to  result  in  reiteration  through  as  many  of  the  preceding 
steps  as  necessary. 


The  focus  of  this  research  will  be  the  decision  process  embedded 
in  the  Preliminary  Activities  phase.  However,  in  order  to  adequately 
address  this  decision  process  it  is  necessary  to  first  establish  the 
relevant  context  by  briefly  describing  the  complete  set  of 
Design/Planning  phases.  The  following  paragraphs  provide  the  needed 
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Production- 

Consumption 

Phases 


Consumption/ 

Operations 


Figure  3.  Asimow/Ostrofsky  Design  Methodology 
(Ostrofsky,  1977d) 


description,  drawing  heavily  on  the  original  work  of  Asimow  (1962)  and 
Ostrofsky  (1977d). 
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Feasibility  Study 

The  purpose  of  the  Feasibility  Study  is  to  establish  a  set  of 
solutions  to  meet  the  stated  needs.  Note  that  this  goal  implies  both 
a  statement  of  needs  and  generation  of  potential  solutions.  The 
specific  steps  required  are  shown  in  Figure  4.  Each  of  the  indicated 
steps  is  described  in  the  following  paragraphs.  Figure  4  formally 
reflects  the  presence  of  iteration  when  new  information  or  insight 
indicates  the  need  to  revisit  previous  decisions. 

Needs  Analysis 

In  this  first  crucial  step,  a  general  statement  of  the  problem  is 
agreed  on  and  documented.  This  requirement  serves  not  only  to  define 
the  direction  of  subsequent  activities,  but  also  to  verify  that  there 
appears  to  be  sufficient  need  to  justify  a  serious  pursuit  of  a 
solution. 

Typically,  some  documentation  concerning  the  need  has  been 
created  prior  to  initiation  of  the  design  effort.  While  this  is 
certainly  an  important  input  to  the  needs  analysis,  it  is  not  the  only 
input.  Experience  indicates  that  written  statements  virtually  never 
contain  a  complete  description  of  the  needs.  Therefore,  it  is 


suggested  that  observation  be  used  as  a  supplemental  source  of 
information.  This  relationship  is  shown  in  Figure  5. 
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The  process  of  examining  and  stating  the  needs  forces  those 
involved  to  evaluate  the  urgency  of  the  needs.  The  result  of  this 
evaluation  should  then  be  compared  to  the  potential  cost  in  time  and 
resources  necessary  to  complete  a  formal  in-depth  design  effort.  In 
other  words,  the  perceived  worth  of  the  resulting  system  must  be  at 
least  as  great  as  the  costs  necessary  to  design  it.  For  simpler 
systems  this  coarse  screen  may  indicate  that  the  magnitude  of 
activities  we  will  describe  is  not  merited.  However,  for  large-scale, 
complex  systems  this  will  virtually  never  be  the  case. 

Identification  and  Formulation  of  the  Problem 

While  the  broad  problem  statement  emerging  from  the  Needs 
Analysis  provides  a  crucial  starting  point  for  the  design  effort,  it 
generally  lacks  the  specific  detail  necessary  for  the  subsequent 
design  actions.  If  activities  proceed  without  further  clarification, 
the  risk  exists  that  the  solution  may  not  be  responsive  to  the  true 
needs.  Therefore,  this  step  strives  to  refine  the  problem  statement 
to  a  greater  level  of  detail. 

The  framework  for  this  effort  is  formed  by  combining  the  system 
inputs/outputs  with  the  system  life  cycle  (Production/Consumption) 
phases  to  form  a  two-dimensional  matrix  whose  cells  require 
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specification.  Recall  that  the  system  inputs/outputs  defined  by 
Asimow  (1962)  are  the  environmental  inputs,  intended  inputs,  desired 
outputs,  and  undesired  outputs.  Further  recall  the  definition  of  four 
life  cycle  phases  subsequent  to  the  design  activity:  production, 
distribution,  operations/consumption,  and  retirement.  Consequently, 
identification  and  formulation  of  the  problem  produces  a  sixteen  cell 
matrix  as  shown  in  Figure  6.  This  is  referred  to  as  the  input-output 
matrix . 

Although  it  is  theoretically  possible  to  complete  the  matrix 
cells  in  virtually  any  order,  experience  and  logic  have  produced  the 
recommended  completion  sequence  shown  in  Figure  7 .  In  this  sequence , 
one  would  first  specify  the  desired  and  undesired  outputs  of  each 
phase  of  the  system  life  cycle.  Next,  the  environmental  inputs  (i.e. 
those  inputs  to  the  system  which  will  exist  without  any  intervening 
efforts)  for  each  phase  are  defined.  Finally,  having  specified  what 
is  desired  from  each  phase  and  what  inputs  (both  good  and  bad)  will  be 
available,  it  remains  only  to  specify  the  missing  input  ingredients 
which  must  be  made  available  to  generate  the  desired  outputs. 

Take  as  a  simple  example  the  planning  of  a  concert  to  be  held  in 
a  public  park.  A  primary  desired  output  in  the  consumption/operations 
phase  might  be  an  entertained  and  satisfied  audience.  An  undesired 
output  might  be  disturbed  residents  in  the  vicinity  of  the  park. 
Environmental  inputs  might  include  the  weather  and  any  existing  park 
facilities.  Based  on  the  inputs  and  outputs  specified  so  far,  it  is 
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DO  ■  Desired  outputs 
UO  *  Undesired  outputs 
El  *  Environmental  inputs 
I!  *  Intended  inputs 


P  *  Production 

D  •  Distribution 

C/O  ■  Consumption/operations 

R  ■  Retirement 


Figure  7.  Sequence  for  Input-Output  Matrix  Completion 

(Ostrofsky,  1977d) 
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clear  that  factors  such  as  a  band,  additional  facilities,  and 
advertising  might  be  specified  as  intended  inputs.  With  respect  to 
the  retirement  phase,  desired  outputs  might  include  safe  and  efficient 
transportation  home  for  the  participants.  Litter  might  be  an 
undesired  output.  Prevailing  traffic  conditions  and  capacities  might 
be  an  important  environmental  input.  Finally,  traffic  directors  and 
cleanup  crews  might  be  perceived  as  necessary  (intended)  inputs  to  the 
concert's  retirement  phase.  The  production  and  distribution  phases 
are  described  in  a  similar  manner. 

Synthesis  of  Solutions 

The  preceding  steps  of  the  Feasibility  Study  were  analytic  in 
nature  since  they  began  with  broad  concepts  and  attempted  to  break 
them  down  into  meaningful  components.  The  result  of  these  efforts  is 
a  detailed  formulation  of  the  problem  to  be  addressed.  The  remaining 
two  steps  of  the  Feasibility  Study  attempt  to  respond  to  these  needs 
by  the  complementary  process  of  synthesizing  alternative  solutions. 
While  analysis  separates  an  entity  into  its  constituent  elements, 
synthesis  is  the  opposite  process  in  which  separate  elements  are 
combined  into  a  unified  entity.  The  purpose  of  the  Synthesis  of 
Solutions  is  to  generate  as  many  potential  design  alternatives 
(candidate  systems)  as  possible  to  meet  the  defined  needs  by  combining 
various  approaches  to  performing  each  required  system  function. 

Given  this  objective,  virtually  any  creative  technique  which 
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encourages  generation  of  alternatives  may  be  employed.  However, 
experience  has  shown  the  following  approach  to  be  useful  for  this 
purpose,  both  because  it  provides  a  well-defined,  perhaps  even 
mechanical  aid  to  creativity  and  because  the  process  helps  insure 
thorough  consideration  of  the  possible  alternatives. 

Synthesis  is  accomplished  by  first  identifying  broad  approaches 
which  accomplish  each  function  required  of  the  final  system.  These 
approaches  are  referred  to  as  "concepts".  It  is  generally  the  case 
that  more  than  one  concept  can  be  identified  to  meet  the  needs .  For 
example,  if  the  stated  need  is  to  establish  some  means  of  passenger 
transportation  between  the  east  and  west  coasts  of  the  United  States, 
one  concept  might  be  an  airplane,  another  might  be  a  bus,  and  a  third 
might  be  a  train. 

Next,  each  concept  is  subdivided  into  the  various  subsystems 
necessary  for  it  to  meet  the  needs.  For  example,  the  train  concept 
might  be  subdivided  into  the  following  subsystems:  passenger 
terminal,  passenger  seating,  dining  facilities,  sleeping  facilities, 
propulsion  (i.e.  locomotive),  etc.  It  is  generally  the  case  that 
there  will  be  more  than  one  means  of  achieving  each  subsystem. 
Therefore,  the  alternatives  are  identified  for  each  subsystem  of  each 
concept  as  shown  in  Figure  8.  Synthesis  of  candidate  systems  then 
becomes  a  straightforward  matter  of  forming  all  possible  combinations 
of  subsystem  alternatives.  If  a  concept  has  three  subsystems,  with 
the  subsystems  having  five,  four,  and  eight  alternatives  respectively, 
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then  a  total  of  160  candidate  systems  (5x4x8-160)  may  be  synthesized. 
This  process  is  repeated  for  each  concept  to  form  the  total  pool  of 
candidate  systems.  (For  the  remainder  of  this  paper  we  will  use  the 
subscript  a  to  represent  a  specific  candidate  system.)  Howard  (1988) 
refers  to  the  representation  of  this  approach  to  the  specification  of 
alternatives  as  the  "strategy  generation  table". 

At  this  point  in  the  design  process,  the  focus  is  on  producing 
the  largest  possible  pool  of  candidate  systems.  The  reason  for  this 
objective  lies  in  the  concept  of  optimization.  Simply  put,  if  we 
speculate  that  for  each  given  design  problem  there  exists  some 
theoretically  best  (optimum)  candidate  system,  then  by  increasing  the 
number  of  candidate  systems  we  identify,  we  may  increase  our  chances 
of  considering  the  optimum  solution.  Unfortunately,  since  we  lack  an 
absolute  benchmark  for  optimality  and  since  we  can  never  be  sure  we 
have  considered  every  possible  candidate  system,  we  will  never  know  if 
we  have  selected  or  even  considered  the  optimum  solution.  Therefore, 
we  will  subsequently  differentiate  by  using  the  term  "optimal"  to 
refer  to  the  candidate  system  we  select. 

It  is  likely  that  some  of  the  candidate  systems  generated  in  this 
manner  may  be  unfeasible  or  undesirable.  While  we  do  not  wish  to 
immediately  eliminate  any  candidates  during  the  synthesis  process  for 
the  reason  just  described,  the  next  step  reduces  the  magnitude  of 
evaluating  alternatives  by  attempting  to  eliminate  such  alternatives 
from  further  consideration. 
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Screening  of  Candidate  Systems 

While  we  have  attempted  to  synthesize  as  many  candidate  systems 
as  possible,  we  do  not  wish  to  waste  resources  evaluating  alternatives 
which  are  clearly  unachievable  or  undesirable.  Therefore,  in  this 
step  we  establish  three  screens  (Figure  9)  to  eliminate  those 
candidate  systems  which  do  not  merit  further  evaluation. 

The  first  screen  is  that  of  physical  realizability.  Candidate 
systems  which  cannot  physically  be  implemented  are  eliminated.  For 
example,  if  one  subsystem  of  a  concept  requires  availability  of  an 
electric  power  source  then  all  candidate  systems  formed  by  pairing 
that  alternative  with  non- electric  power  sources  might  be  deemed 
physically  unachievable  and  eliminated.  Recognizing  that  large  design 
efforts  are  likely  to  span  long  periods  of  time,  this  determination  is 
not  made  based  solely  on  current  capabilities,  but  also  under 
consideration  of  capabilities  which  might  be  achieved  within  the 
planning  horizon. 

The  second  screen  concerns  economic  worthwhileness.  This  screen 
eliminates  all  candidate  systems  whose  potential  value  clearly  does 
not  exceed  the  resources  necessary  to  achieve  it.  Traditional 
techniques  such  as  breakeven  analysis  may  be  useful  in  this  regard. 
Note,  however,  that  the  determination  of  value  may  be  made  in  terms  of 
any  measures  deemed  appropriate  and  is  not  necessarily  restricted  to 


monetary  considerations.  The  objective  is  simply  to  establish  rough 
preliminary  assessments  of  the  tradeoffs  between  the  costs  of 
implementation  and  the  benefits  to  be  derived.  While  this  examination 
is  similar  to  the  tradeoff  evaluation  implicitly  generated  initially 
in  the  Needs  Analysis,  it  goes  much  farther.  The  earlier  assessment 
would  only  have  discouraged  continuation  if  the  projected  benefits 
could  not  even  exceed  the  cost  of  the  design  effort  itself.  Here,  a 
more  selective  evaluation  is  made  with  respect  to  individual  candidate 
systems  and  the  potential  costs  considered  include  all  those  which 
would  accrue  throughout  the  system's  life  cycle. 

The  third  screen  concerns  financial  feasibility.  This  screen 
simply  asks  whether  the  minimum  resources  necessary  to  achieve  the 
candidate  system  are  or  could  become  available.  If  not,  the  candidate 
is  eliminated  from  consideration. 

At  this  early  stage  of  the  design,  process,  little  factual  data 
may  be  available  concerning  the  candidate  systems.  Therefore,  the 
physical,  economic,  and  financial  screens  are  likely  to  be  fraught 
with  uncertainty.  To  govern  the  decision  in  such  cases,  the  pervasive 
rule  is  established  that  no  candidate  system  is  eliminated  unless  it 
is  clearly  unachievable,  uneconomical,  or  infeasible .  In  other  words , 
if  in  doubt,  the  candidate  system  should  not  be  eliminated  in  order  to 
retain  the  highest  possibility  that  the  optimum  has  been  included. 
The  tradeoff  between  the  desire  to  find  the  optimum  by  maximizing  the 
size  of  the  set  of  candidate  systems  and  the  corresponding  difficulty 


in  thoroughly  evaluating  a  large  set  of  alternatives  is  referred  to  as 
the  "designer's  dilemma." 
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Using  the  set  of  candidate  systems  synthesized  during  the 
Feasibility  Study,  the  purpose  of  the  Preliminary  Activities  becomes 
the  selection  of  the  optimal  candidate  system:  that  is,  the  defined 
candidate  system  which  will  best  satisfy  the  stated  needs. 


Preparation  for  Analysis 


Since  large-scale  design  efforts  frequently  span  a  long  period  of 
time,  it  is  not  unusual  for  temporal  breaks  to  occur  between  major 
portions  of  the  design  activities.  Such  breaks  may  occur  to  permit 
formal  reviews,  funding  decisions,  or  organizational  and  staffing 
efforts  for  the  more  resource- intensive  efforts  to  follow.  The 
proposed  methodology  recognizes  the  need  for  such  breaks  by  including 
a  preparation  step  at  the  start  of  the  second  major  phase.  The 
purpose  of  this  step  is  to  provide  an  opportunity  for  designers  to 
refamiliarize  themselves  with  the  results  of  the  activities  which  have 
gone  before  and  adjust  to  changes  required  by  improvements  revealed 
subsequently  to  original  decisions.  Although  the  rigor  with  which 
this  step  is  accomplished  will  vary  based  on  the  nature  of  the  effort 
and  the  time  which  has  elapsed  since  the  Feasibility  Study,  it 


provides  a  logical  basis  for  educating  new  project  personnel  as  the 
design  effort  gradually  expands  beyond  the  smaller  cadre  typically 
involved  in  the  early  activities . 

Since  the  primary  focus  of  this  step  is  on  familiarization,  the 
recommended  approach  is  to  reexamine  the  synthesized  set  of  candidate 
systems,  grouping  them  according  to  their  common  characteristics.  The 
input-output  matrix  serves  as  a  basis  for  this  grouping.  Although 
there  is  no  formal  output  from  this  step,  the  increased  awareness  of 
the  characteristics  of  the  candidate  systems  and  the  stated  needs 
serves  as  a  basis  for  subsequent  analytic  activities.  It  is  also 
possible  that  the  process  may  highlight  the  need  for  iteration, 
especially  if  a  substantial  period  of  time  has  elapsed  since  the 
completion  of  the  Feasibility  Study. 

Definition  of  Criteria  and  Relative  Weights 

Since  the  purpose  of  the  Preliminary  Activities  is  the  selection 
of  the  optimal  candidate  system,  it  is  necessary  that  decision 
criteria  be  established.  Simply  stated,  criteria  are  the  standards 
against  which  the  performance  of  the  system  will  be  evaluated. 
Criteria  are  normally  characteristics  which  are  broad  enough  to 
preclude  direct  measurement,  e.g.  safety,  availability,  performance, 
and  even  cost.  Although  there  is  typically  broad  implicit  agreement 
on  the  relevant  criteria,  their  formal  statement  is  essential  since 
quantitative  mechanisms  will  be  used  for  the  evaluation  process. 


Therefore,  any  criterion  not  expressly  considered  will  not  be  included 
in  the  selection  of  the  optimal  candidate  system. 

At  this  stage  there  may  be  a  temptation  to  specify  each  detailed 
characteristic  that  the  envisioned  system  should  have.  This 
temptation  should  be  resisted,  however,  since  subsequent  steps  in  the 
methodology  will  be  used  to  "flesh  out"  the  relevant  details. 
Instead,  this  step  is  intended  to  highlight  the  broad  inclusive 
qualities  desired. 

Once  a  set  of  criteria  (denoted  {x^})  has  been  established,  each 
criterion  is  assigned  a  relative  weight  (denoted  {aj_}).  This  action 
is  based  on  the  generally  applicable  case  where  not  all  criteria  are 
of  equal  importance.  For  reasons  which  will  be  shown  later,  the 
relative  weights  are  defined  such  that 

n 

Sa^-0  Eq .  2.1 

i-1 

where  the  subscript  i  refers  to  the  ith  criterion. 

A  variety  of  methods  have  been  established  in  the  literature  to 
facilitate  the  derivation  of  relative  weights  (Churchman,  1957; 
Fishburn,  1964;  Huber,  1974b).  A  comparison  of  these  approaches  is 
beyond  the  scope  and  intent  of  this  research.  For  our  purposes,  any 
method  is  acceptable  which  yields  weights  reflecting  the  desired 
objectives  of  designers,  producers,  users,  etc.  Given  the 


46 


multiplicity  of  relevant  parties,  a  method  which  explicitly 
accommodates  group  preferences  (e.g.  Delphi,  group  decision  support 
systems,  etc.)  would  normally  be  desirable.  In  the  simplest  case  of 
constant  relative  weights,  the  resulting  description  of  criteria  might 
look  like  the  example  in  Table  1. 

Definition  of  Parameters 

While  the  previous  step  identified  the  broad  characteristics  of 
interest,  many  of  these  attributes  will  not  be  directly  measurable. 
Therefore,  in  this  step,  directly  measurable  characteristics 
(parameters)  are  identified  from  the  set  of  candidate  systems  which 
can  later  be  combined  to  represent  the  defined  criteria.  This  process 
is  achieved  as  follows. 

For  each  criterion,  a  brainstorming  effort  is  undertaken  to 
identify  an  exhaustive  set  of  constituent  elements  for  that  criterion. 
These  elements  are  then  listed  in  tabular  form  and  coded  (Table  2)  to 
indicate  whether  each  element  can  be: 

a)  directly  measured, 

b)  measured  from  some  model  that  includes  other  elements  coded  as 
a' s  or  b' s , 

c)  completely  included  in  other  elements,  or 

d)  not  measurable  within  existing  resources  (or  more  accurately, 
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TABLE  1. 

CRITERIA  WITH  RELATIVE  WEIGHTS 
(Ostrofsky,  1977d) 


Critstiotk^Xi 

x^ ,  play  value 

X2 ,  educational  value 

X3 ,  production  cost 

x4,  durability 

X5 ,  quality 


Weight,  a^ 

&1  -  4,  4/20  -  .20 
a2  -  3,  3/20  -  .15 
a3  -  5,  5/20  -  .25 
a4  -  2,  2/20  -  .10 
a5  6/20 


20 


1.00 


TABLE  2. 


CRITERION  ELEMENTS 
(Ostrofsky,  1977d) 


Criterion:  Play  value 

Elements  Code 

Enj  oyment  b 

Interest  b 

Number  of  colors  a 

Number  of  moving  parts  a 

Number  of  instructions 

required  to  operate  a 

Assembly  time  a 

Operating  time  a 

Style  b 

Density  b 

Number  of  different 

types  of  surface  a 

Toy  volume  a 

Toy  weight  a 

Number  of  ways  of  moving  a 

Internal  volume  a 

Fascination  d 


resources  planned  to  be  available  within  the  timeframe  of  the  design 
effort) . 
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The  goal  of  this  process  is  to  be  able  to  define  all  constituent 
elements  as  a,  b,  or  c  so  that  they  may  be  included  in  the  evaluation 
process.  If  items  are  coded  as  d,  they  must  be  individually  examined 
to  see  if  the  importance  of  their  consideration  merits  additional 
efforts  to  obtain  their  measurement,  even  using  subjective  scales  or 
proxy  attributes  (Keeney  and  Raiffa,  1976:  55)  if  necessary. 

Since  elements  coded  as  c  (completely  included  in  other  elements) 
need  not  be  explicitly  considered  any  further,  the  task  remains  to 
relate  those  elements  coded  as  a  and  b.  Recall  that  these 
designations  indicate  that  the  elements  can  be  directly  measured  or 
measured  as  a  model  of  the  other  elements,  respectively.  The  elements 
coded  "a"  are  termed  parameters  (denoted  {y^} )  and  the  "b"  elements 
are  designated  as  submodels  (denoted  { z j ) ) .  A  table  such  as  Table  3 
is  constructed  to  relate  the  parameters  and  submodels  to  each  other. 
Note  that  a  submodel  will  typically  be  measured  from  more  the  one 
parameter  and  that  parameters  may  be  included  as  inputs  to  more  than 
one  submodel.  Also  note  that  submodels  may  serve  as  the  inputs  to 
other  submodels . 

Given  the  relationships  between  parameters  and  submodels,  it 
remains  to  relate  both  to  their  parent  criteria.  This  process  may  be 
facilitated  using  a  table  such  as  that  shown  in  Table  4.  Note  that 


TABLE  3. 


RELATION  OF  PARAMETERS  TO  SUBMODELS 
(Ostrofsky,  1977d) 


Submodels , 

Zj  (b) 

E 

I 

S  D 
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t  e 

J 

t 

y  n 
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e 

1  s 

y 

r 

e  i 
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n 

t 

Parameters,  (a) 

t 

Number  of  colors 

X 

X 

X 

Number  of  moving  parts 

X 

X 

Number  of  instructions 


required  to  operate 

X 

X 

Assembly  time 

X 

X 

Operating  time 

X 

X 

Number  of  different 

types  of  surface 

X 

X 

X 

Total  volume 

X 

X 

X 

Total  weight 

X 

X 

Number  of  ways  of  moving 

X 

X 

Internal  volume 


X 


TABLE  4. 


RELATION  OF  PARAMETERS  AND  SUBMODELS  TO  CRITERIA 


Parameters,  y^  (a) 


(Ostrofsky,  1977d) 

Criteria, 

_xi -  — ^2  - —3 — - 


-S4- 


Number  of  colors 
Number  of  moving  parts 
Number  of  instructions 

required  to  operate 
Assembly  time 
Operating  time 
Number  of  different 

types  of  surface 
Total  volume 
Total  weight 

Number  of  ways  of  moving 
Internal  volume 
Number  of  lessons  covered 
Number  of  toys  produced 
Cost  of  material  per  toy 
Number  of  nonmoving  parts 
Part  thickness 
Temperature  allowed 


Submodels , 
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X  X! 


this  format  highlights  the  possibility  of  parameters  relating  not  only 
to  more  than  one  submodel,  but  also  to  more  than  one  criterion. 
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Therefore,  it  is  entirely  plausible  that  a  submodel  for  one  criterion 
is  based  on  parameters  relevant  to  a  number  of  other  criteria.  It  is 
also  important  to  note  that  there  is  no  limit  to  the  number  of  levels 
of  submodels . 

During  this  identification  process ,  numerous  opportunities  for 
error  exist.  Therefore,  three  formal  tests  have  been  identified: 
completeness,  compactness,  and  consistency  (Figure  10). 

The  test  for  completeness  requires  that  each  criterion  be 
examined  to  ensure  that  all  parameters  and  submodels  required  for  its 
evaluation  have  been  identified.  As  is  the  case  with  the  criteria 
themselves,  any  element  not  considered  will  not  be  included  in  the 
determination  of  the  optimal  candidate  system. 

Checking  for  compactness  refers  to  insuring  that  only  those 
parameters  and  submodels  necessary  have  been  included.  Since  data 
will  have  to  be  gathered  from  every  candidate  system  for  each 
parameter,  the  size  of  the  parameter  set  is  obviously  related  to  the 
time,  effort,  and  cost  of  the  evaluation  process.  Therefore, 
reasonable  opportunities  to  economize  the  set  of  parameters  should  be 
pursued.  For  example,  in  some  cases,  a  direct  measurement  may  exist 
which  can  eliminate  the  need  for  multiple  parameters  (e.g.  pressure 
vs.  force  and  area). 
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Finally,  testing  for  consistency  means  checking  to  insure  that  a 
parameter  (or  submodel)  has  the  same  meaning  each  time  it  is  used. 
Frequently,  synonyms  or  closely  related  characteristics  may  have 
initially  been  identified  as  necessary  for  the  determination  of  a 
particular  criterion  when,  in  reality,  a  different  attribute  is  really 
required.  Table  4  provides  a  handy  means  of  highlighting  such 
discrepancies . 


Criterion  Modeling 

This  step  links  parameters ,  submodels ,  and  criteria  by 
establishing  quantitative  models  which  relate  them  to  each  other.  In 
many  cases,  theoretically  suggested  and/or  practically  established 
relationships  may  exist  (e.g.  inventory  models,  physical 
relationships,  experimental  observation,  etc.).  In  other  cases,  a 
more  subjective  process  of  establishing  relationships  must  be 
undertaken.  In  the  latter  case,  it  is  typically  useful  to  begin  by 
establishing  functional  relationships  such  as  those  shown  in  Figure 
11.  Given  the  general  nature  of  the  perceived  relationship,  it 
becomes  an  easier  (though  not  necessarily  easy)  process  to  establish 
specific  relationships  via  trial  and  error  or  sensitivity  analysis. 
These  relationships  may  require  field  or  laboratory  testing  for  their 
initial  determination  or  verification. 


In  the  case  where  established  quantitative  relationships  are 
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unavailable,  there  may  be  reluctance  to  speculate  on  the  nature  of 
those  relationships.  The  analyst  then  assumes  the  task  of 
reemphasizing  that  factors  not  considered  will  not  be  included  in  the 
selected  candidate  system.  If  a  characteristic  is  important  enough  to 
be  considered  it  must  be  formulated  to  the  best  extent  practical  even 
to  the  development  of  field  tests  to  provide  data.  This  decision 
defines  the  meeting  ground  between  theory  and  practice  and  is 
precisely  the  issue  which  differentiates  management  from  pure  science. 
Whereas  the  scientist  may  confine  himself  to  factors  which  are 
provable,  the  manager  must  do  the  best  he  or  she  can  with  the 
resources  available .  A  useful  source  of  encouragement  in  this  regard 
is  the  consideration  of  the  impact  of  not  specifying  such  a 
relationship.  It  is  also  worthwhile  noting  that  in  the  absence  of  a 
quantitative  representation  managers  are  still  likely  to  consider 
important  factors,  but  in  an  even  less  structured  and  more  subjective 
manner.  Therefore,  the  goal  becomes  that  of  applying  as  much 
objectivity  as  possible  to  an  inherently  subjective  situation  (Keeney 
and  Raiffa,  1976:  45).  Formal  consideration  of  such  relationships  at 
least  makes  the  nature  of  the  consideration  explicit,  forces 
discussion,  and  facilitates  revision  if  necessary. 

The  resulting  models  should  be  constructed  such  that  criterion 
values  increase  with  higher  numerical  values.  In  other  words,  the 
higher  the  value  of  the  model  output,  the  better  the  performance  of 
the  candidate  system.  Clearly,  however,  the  output  of  each  model  will 
likely  be  in  units  of  measure  not  compatible  with  those  of  other 
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models.  Therefore,  it  remains  for  the  decision  model  to  provide  a 
means  of  combining  or  comparing  those  outputs. 

Formulation  of  the  Criterion  Function  (CF) 

Simon  (1977)  defines  a  criterion  function  as  "a  measure  to  be 
used  for  comparing  the  relative  merits  of  various  possible  courses  of 
action."  In  this  step  of  the  methodology,  the  criteria  measures 
generated  above  are  weighted  and  combined  with  each  other  in  a  manner 
consistent  with  the  values  of  the  design  decision  maker(s) .  This 
process  requires  an  evaluation  of  the  possible  ranges  of  values, 
normalization  of  the  criterion  values,  and  integration  of  the  relative 
weights . 

It  is  likely  that  each  candidate  system  may  be  capable  of 
assuming  any  value  within  a  range  for  some  parameters.  Therefore,  a 
determination  is  made  of  the  minimum  and  maximum  values  for  each 
parameter  for  each  candidate  system.  Overall  parameter  ranges  are 
then  determined  by  taking  the  minimums  and  maximums  over  the  complete 
set  of  candidate  systems.  In  addition,  user-defined  constraints  may 
be  established  for  parameter  value  ranges.  It  may  be  necessary,  for 
example,  to  ensure  that  various  parameter  values  do  not  exceed  or  fall 
below  certain  levels.  These  constraints  are  referred  to  as  "regional 
constraints".  Constraints  which  result  from  physical  or  logical 
relationships  (i.e.  via  submodels)  are  termed  "functional 
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constraints".  Candidate  systems  whose  parameter  ranges  violate 
regional  constraints  may  be  eliminated  from  further  consideration. 

Given  the  parameter  ranges  and  the  quantitative  relationships  of 
the  submodels ,  minimums  and  maximums  may  be  determined  for  each 
submodel  and,  subsequently,  for  each  criterion.  The  ultimate  goal  of 
this  quantification  effort  is  the  production  of  values  which  are 
directly  comparable  across  criteria.  A  simple  means  to  this  end  might 
be  simple  normalization  of  the  form: 

X  “  (x  -  Xmin)/ (^ax  “  ^min^  Eq.  2.2 

However,  it  is  important  to  recognize  that  this  relationship  is 
limited  by  its  implication  of  assumed  linearity  of  importance 
throughout  the  range  of  criterion  values .  For  this  and  other  reasons , 
subsequent  portions  of  this  research  will  show  that  use  of  the 
mathematics  of  probability  theory  is  the  preferred  method  of 
normalization. 

Having  achieved  comparability  across  criteria,  it  remains  to 
include  consideration  of  the  relative  weights  in  obtaining  a  single 
figure  of  merit  for  each  candidate  system.  This  is  typically  achieved 
via  the  well-known  additive  weighting  method,  referred  to  here  as  the 
criterion  function: 


CFq  -  Sa^i 


Eq.  2.3 
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Given  the  form  of  CFa,  the  resulting  values  will  range  between  0 
and  1.0. 


Analysis  of  the  Parameter  (Design)  Space 

It  is  possible  to  conceive  of  a  theoretical  hyperspace  which  has 
an  axis  for  each  identified  parameter  plus  one  axis  for  the  resulting 
value  of  CF.  This  has  been  defined  as  the  "design  space"  (Ostrofsky, 
1977d) .  The  possible  parameter  values  and  their  relationships  to 
criterion  values  via  the  submodels  may  then  be  perceived  to  produce  a 
surface  in  the  hyperspace  consisting  of  the  parameter  and  CF  values  of 
all  theoretically  possible  candidate  systems.  This  surface  may  well 
contain  discontinuities  resulting  from  the  functional  constraints . 
Each  defined  candidate  system  is  represented  by  a  point  on  the  surface 
and  an  associated  vector  of  values,  one  value  for  each  parameter  axis. 
Points  on  the  surface  not  represented  by  a  defined  candidate  system 
may  represent  alternatives  not  previously  considered  or  may  simply 
represent  unachievable  combinations  of  parameter  values. 

Analysis  of  this  hyperspace  is  undertaken  to  reveal  useful 
information  about  the  sensitivity,  compatibility,  and  stability  of  the 
candidate  systems  as  well  as  indications  of  system  performance.  A 
variety  of  computer  search  techniques  exist  from  research  in  other 
applications  (Box  and  Draper,  1986;  Fletcher  and  Powell,  1963; 
Fletcher  and  Reeves,  1964;  Hooke  and  Jeeves,  1961;  Leon,  1966; 
Taubert,  1968;  Shestakov,  1983)  which  may  be  used  to  conduct  this 
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design  space  search.  Sensitivity  in  this  context  refers  to  analysis 
which  searches  for  those  parameters  which  have  the  greatest  impact  on 
the  resulting  value  of  CF.  By  contrast,  compatible  parameters  are 
defined  as  those  which  have  the  least  impact  on  CF.  Stability 
analysis  refers  to  the  identification  of  parameter  value  changes  which 
may  produce  drastic  or  unexpected  consequences .  Recognizing  that 
evaluation  of  candidate  systems  at  this  stage  is  still  based  on 
estimates,  unstable  parameter  combinations  may  represent  risky 
candidate  systems. 


Formal  Optimization 

In  this  step,  optimization  of  CF  is  pursued  based  on  information 
gained  in  the  analysis  of  the  parameter  space.  First,  each  candidate 
system  is  optimized  by  selecting  the  combination  of  permissible 
parameter  values  which  maximizes  its  value  of  CF.  This  process  is 
referred  to  as  optimization  "within"  candidate  systems.  Next,  the 
optimal  candidate  system  is  identified  as  the  candidate  system  with 
the  highest  value  of  CF.  Given  the  construct  of  CF,  that  candidate 
system  will  be  the  one  whose  CFQ  is  closest  to  (but  not  greater  than) 
1.0.  This  determination  is  termed  optimization  "among"  candidate 
systems.  Although  there  is  theoretically  only  one  optimal  candidate 
system,  it  is  quite  possible  that  many  candidate  systems  will  have  CF 
values  nearly  equally  close  to  1.0.  If  the  degree  of  accuracy  of 
measurements  is  less  than  the  magnitude  of  difference,  then  for  all 
practical  purposes  these  candidates  are  equally  desirable.  In  such  a 


.  case ,  other  factors  may  be  used  to  select  the  preferred  alternative  or 
more  accurate  data  may  be  sought. 
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As  stated  previously,  it  is  recognized  that  the  assessment  of 
parameter  ranges  for  each  candidate  system  may  have  been  based  on 
incomplete  information.  Therefore,  it  is  possible  that  subsequent 
design  activities  may  mandate  changes  in  the  achievable  parameter 
values  for  a  particular  candidate  system.  Consequent  changes  in  the 
system's  parameter  values  clearly  affect  the  resulting  value  of  CF. 
Since  the  candidate  chosen  was  selected  based  on  its  best  combination 
of  parameter  values,  any  change  in  those  values  will  result  in  a 
decrease  in  CF  if  the  original  work  was  accurate.  Therefore,  when 
alternative  changes  are  possible,  it  will  probably  be  preferable  to 
use  the  results  of  the  analysis  of  the  parameter  space  to  insure 
changes  are  made  to  compatible  (less  sensitive)  parameters  rather  than 
sensitive  ones.  Even  so,  the  resulting  degradation  of  CFa  may  result 
in  another  candidate  system  defaulting  to  become  the  optimal.  In  this 
case,  theory  would  indicate  that  the  design  process  should  Iterate 
through  all  intervening  steps  using  the  new  optimal  candidate  system. 
However,  it  is  recognized  that  in  practice  this  may  be  infeasible. 
Therefore,  the  new  optimal  candidate  system  instead  becomes  a  target 
for  modification  efforts  or  the  next  design  effort. 
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Projection  of  System  Behavior 

The  optimal  candidate  system  is  analyzed  more  extensively  to 
examine  and  predict  its  ultimate  behavior  on  characteristics  of 
interest.  Simulation  and/or  appropriate  analytical  techniques  may  be 
used  for  these  purposes.  As  a  minimum,  the  projected  performance  of 
the  optimal  candidate  system  is  examined  in  the  context  of  each  of  the 
four  production/consumption  phases  of  the  system  life  cycle.  This 
step  also  provides  the  best  opportunity  encountered  so  far  in  the 
design  process  to  obtain  detailed  cost  estimates  for  the  selected 
system  throughout  its  life  cycle. 

Testing  and  Simplification 

While  the  preceding  step  provides  more  detailed  information  than 
previously  available,  those  projections  are  still  largely  speculative. 
Therefore,  testing  is  warranted  to  verify  the  system's  attributes  in 
order  to  reduce  the  risks  of  estimation.  Testing  may  be  used  to 
verify  hypotheses,  generate  new  information,  and  expose  shortcomings. 

As  an  important  by-product  of  the  testing  process,  recognition  is 
frequently  gained  of  areas  in  which  the  design  may  be  simplified  or 
streamlined.  Therefore,  needless  complexities  may  be  removed  and 
simplifications  introduced  as  long  as  system  performance  remains 
comparable  or  improves . 


Collectively,  these  steps  are  uniquely  tailored  to  anticipate  and 
resolve  every  eventuality  of  the  production/consumption  phases  by 
preparing  for  the  implementation  and  sustained  operation  of  the 
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system.  Consequently,  the  specific  activities  undertaken  are 
determined  by  the  nature  of  the  system  being  designed  and  the 
environment  in  which  it  is  to  be  produced,  distributed,  operated,  and 
retired.  Such  activities  might  include,  for  example,  preparation  of 
technical  data  (e.g.  blue  prints,  specifications),  design  of  support 
equipment,  plans  for  distribution,  training,  operator's  manuals, 
maintenance  specifications,  replacement  parts  acquisition 
requirements,  disposition  instructions,  etc.  The  detailed  activities 
associated  with  micro- level  design  and  development  approaches  would 
normally  fall  within  this  category.  In  general,  however,  a  plan  is 
developed  to  accomplish  all  activities  required  for  each  phase  in  the 
production- consumption  phases. 


Summary 

The  essence  of  a  design  methodology  closely  parallels  Simon's 
(1977)  classic  stages  of  management  decision  making  with  the  appendage 
of  an  implementation  phase  (Hogarth  and  Makridakis,  1981b): 
intelligence  (the  awareness  of  the  problem  or  need),  design  (the 
synthesis  of  solutions),  choice  (the  selection  of  a  specific  candidate 
system),  and  implementation  (the  detail  activities).  The  focus  of 
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this  research  is  the  development  of  a  decision  making  approach  (the 
choice  stage)  within  the  context  of  design  methodology.  The  specific 
methodology  presented  in  this  chapter  was  selected  as  a  framework  for 
the  balance  of  the  research  because  it  has  provided  a  sound  framework 
for  previous  research  (Ostrofsky  et  al,  1977c,  1978,  1979,  1980,  1981; 
Folkeson,  1982;  Wu,  1983;  Peschke,  1986),  because  it  embodies  the 
significant  developments  in  the  area  of  design  methodology  as 
described  previously,  and  because  it  provides  the  inputs  required  by 
the  proposed  decision  process. 

The  methodology  is  composed  of  the  three  Design/Planning  phases 
of  the  Feasibility  Study,  the  Preliminary  Activities,  and  the  Detail 
Activities.  First  developed  by  Asimow  (1962),  these  phases  have  been 
proliferated  by  other  authors  and  closely  parallel  alternative 
groupings.  The  methodology  expressly  considers  the  four  system  life 
cycle  phases  of  production,  distribution,  operations/consumption,  and 
retirement  in  its  identification  and  formulation  of  the  problem  and 
its  projection  of  system  behavior.  These  descriptors  of  a  system's 
life  cycle  appear  to  be  the  accepted  standard.  The  methodology  takes 
a  decidedly  macro -level  vi»w  of  the  design  effort,  utilizes  an 
inductive  approach  in  its  synthesis  of  candidate  systems,  and  requires 
an  expansionist  perspective  in  its  modeling  and  optimization 
activities . 

As  a  basis  for  the  target  decision  process,  the  methodology  makes 
available  virtually  all  of  the  required  inputs.  It  acknowledges  the 
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complexity  of  the  decision  environment  both  in  its  synthesis  of  the 
largest  possible  set  of  candidate  systems  and  in  its  consideration  of 
multiple,  potentially  conflicting  objectives.  Its  criterion  function 
pursues  a  prescriptive  approach  to  decision-making  in  which 
optimization  is  sought.  Subjective  considerations  are  formally 
included  both  during  the  determination  of  relative  weights  and  in  the 
modeling  process.  The  use  of  relative  weights  also  recognizes  the 
difference  in  importance  of  decision  criteria.  Finally,  while  the 
description  to  this  point  has  not  discussed  the  treatment  of  criteria 
interactions,  it  is  at  least  clear  that  the  methodology  permits  and 
recognizes  the  presence  of  interaction  in  the  specification  of 
criteria,  parameters,  and  submodels.  The  subsequent  chapters  of  this 
research  are  concerned  with  the  pursuit  of  a  decision  making  process 
which  integrates  all  of  these  various  considerations  in  the  best 
possible  manner. 


Chapter  3 


QUANTITATIVE  APPROACHES  TO  DECISION  MAKING 

The  terms  "problem  solving"  and  "decision  making"  are  frequently 
used  interchangeably.  However,  Smith  (1988)  has  provided  the 
following  useful  distinction.  Problem  solving  concerns  the 

achievement  of  a  goal  or  the  movement  from  some  unsatisfactory  state 
to  a  more  desirable  one.  By  contrast,  decision  making  implies  that  a 
choice  must  be  made  among  alternatives  which  have  been  (or  will  be) 
defined.  This  research  is  concerned  with  decision  making:  the  choice 
among  candidate  systems. 

Eilon  (1985)  has  further  differentiated  four  types  of  decision 
making : 

1)  decision  making  under  deterministic  conditions:  well  defined 
goals,  information  fully  available,  deterministic  variables  and 
outcomes , 

2)  decision  making  under  risk:  well  defined  goals,  information 
fully  available,  stochastic  variables, 

3)  decision  making  under  uncertainty:  goals  are  well  defined 
but  information  is  incomplete, 

4)  decision  making  under  ambiguity:  goals  are  unclear, 
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information  is  incomplete . 

In  most  cases,  we  might  expect  the  design  environment  to  at  least 
require  decision  making  under  uncertainty  since  information  about  the 
candidate  systems  is  generally  estimated  and/or  projected  based  on 
past  experience.  As  Eilon  (1985)  describes,  under  uncertain 
conditions  "...  information  about  future  events  is  incomplete  ..." 
and  ".  .  .  for  a  given  set  of  assumptions  and  performance  criteria,  a 
range  of  possible  outcomes  can  be  computed  .  .  . "  Even  in  the  best 
(theoretical)  possible  case  where  information  about  the  candidate 
systems  is  complete  and  the  designer  faces  decision  making  under 
deterministic  conditions,  Eilon  (1985)  observes  that  "these  conditions 
do  not  necessarily  render  problems  in  this  category  simplistic  or 
trivial"  and  that  many  such  problems  (>  g.  the  travelling  salesman 
problem),  while  highly  structured  and  deterministic,  are  nevertheless 
complex  and  difficult  to  solve. 

The  presentation  of  the  design  methodology  in  Chapter  2  included 
a  brief  description  of  one  approach  (the  criterion  function)  to  design 
decision  making.  However,  a  great  deal  of  research  concerned  with 
decision  making  is  available  which  might  be  useful  in  pursuit  of  the 
stated  research  goals  (Blin,  1977;  Bridgema-i,  1922;  Carlsson,  1983; 
Cheng  and  Mclnnis ,  1980;  Cohen  et  al,  1985;  Epstein,  1957;  Green  and 
Srinivasan,  1978;  Harrison,  1975;  Holloway,  1979;  Ignizio,  1976;  Janis 
and  Mann,  1977;  Keeney,  1982;  Miller,  19701  Roy,  1977;  Rubinstein, 
1975,  1980,  1986;  Sage,  1977;  Sakawa  and  Seo,  1980;  Shull,  Delbacq, 
and  Cummings,  1970;  Simon,  1960,  1978;  Tribus,  1969;  White  and  Sage, 


1980;  Wymore,  1976;  Yager,  1981;  Zeleney,  1973,  1974,  1976a,  1976b, 
1977,  1980,  1981a,  1981b,  1982,  1984).  The  purpose  of  this  chapter  is 
to  review  some  of  the  primary  existing  decision  making  techniques 
which  are  currently  used  or  could  be  used  to  facilitate  the  type  of 
decision  making  stated  as  the  objective  of  this  research.  This  review 
is  intended  to  highlight  the  strengths  and  weaknesses  of  approaches 
commonly  suggested  as  most  promising  for  this  type  of  decision 
problem. 

Two  types  of  approaches  will  be  reviewed:  generic  quantitative 
approaches  and  approaches  which  have  been  pursued  specifically  for  the 
design  decision  environment.  Each  technique  will  be  described  both  in 
general  terms  and  in  the  context  of  the  design  methodology  framework. 
Finally,  a  summary  review  of  the  limitations  of  the  existing 
approaches  will  be  presented  to  suggest  the  preferred  direction  for 
the  remainder  of  this  research. 

General  Quantitative  Approaches 

Many  solution  techniques  have  been  developed  since  World  War  II 
within  the  field  of  operations  research  (Wagner,  1975) .  Originally 
conceived  to  address  specific  interdisciplinary  problems,  the  most 
powerful  of  these  approaches  have  been  generalized  to  permit 
application  to  broad  classes  of  problems  which  lend  themselves  to 
common  formulations.  We  will  consider  three  such  general  approaches: 
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mathematical  programming,  decision  analysis,  and  dimensionless 
analysis.  Although  there  are  many  forms  of  mathematical  programming, 
we  will  confine  our  discussion  to  the  two  most  frequently  suggested 
for  applications  such  as  those  considered  in  this  research:  linear 

programming  and  goal  programming. 

Linear  Programming 

Linear  optimization  approaches  have  been  described  as  being  among 
the  most  commercially  successful  applications  of  operations  research 
techniques  (Wagner,  1975).  In  general  terms,  linear  programming 
(L.P.)  attempts  to  find  the  mix  of  decision  variable  values  which 
maximizes  (or  minimizes)  some  objective  function  or  overall  measure  of 
value  subject  to  a  set  of  constraints  on  the  values  the  decision 
variables  may  assume.  Common  applications  have  included  product  mix 
decisions,  production  scheduling,  and  transportation  or  routing 
problems.  Specific  solution  approaches  have  been  developed  for 
special  cases  such  as  transportation  and  assignment  problems  as  well 
as  for  the  general  case . 

The  general  form  of  the  L.P.  (maximization)  problem  is: 

n 

max  z  -  Sc <X4  Eq.  3.1 

j-1 

n 

Sa^4X4  <  b^  for  i-1,2, . . . ,m  Eq.  3.2 
j-1 


subject  to 
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Xj  >  0  for  j-l,2,...,n  Eq.  3.3 

In  this  maximization  formulation,  z  is  the  objective  function 
value,  the  Xj  are  the  decision  variables,  and  the  cj  are  the  per  unit 
benefits  associated  with  the  Xj .  The  first  series  of  constraints  are 
sometimes  referred  to  as  resource  constraints.  In  each  of  the  m 
constraints,  the  a^j  are  coefficients  which  form  a  linear  combination 
of  the  decision  variables  (xj)  which  is  limited  by  some  value  b^. 
Since  the  a^j  and  bj_  may  assume  any  value  and  m  may  be  any  positive 
integer,  the  constraint  set  may  be  of  virtually  any  size  or  nature  in 
the  general  form.  The  special  cases  mentioned  above  are  applicable 
when  the  constraint  set  assumes  specific  characteristics.  The  second 
series  of  constraints  forces  the  decision  variables  to  be  non¬ 
negative:  a  generally  applicable  requirement  when  the  variables 
represent  the  amount  of  some  physical  substance  or  characteristic. 

Note  that  the  formulation  of  the  L.P.  problem  reflects  its 
inherent  applicability  and  assumptions.  The  foremost  distinguishing 
characteristic  is  its  assumption  of  linearity  in  both  the  objective 
function  and  the  constraints.  Although  this  assumption  may  not  be 
strictly  true  in  many  cases,  it  is  frequently  close  enough  to  produce 
reasonable  solutions,  particularly  when  weighed  against  the  difficulty 
of  non-linear  solution  techniques.  Closely  related  to  the  assumption 
of  linearity  is  the  characteristic  of  a  continuous  range  of  values 
which  the  decision  variables  may  assume.  Other  L.P.  formulations  and 
solution  techniques  are  available  when  integer  restrictions  apply; 


71 

however,  the  associated  computational  complexity  of  these  additional 
restrictions  can  be  substantial. 


It  is  convenient  to  visualize  the  L.P.  problem  and  its  solution 
graphically  for  the  case  of  two  decision  variables.  Assume  the 
example  problem: 


subject  to 


7x^  +  10x2 

Eq.  3.4 

3x^  +  2x2  —  36 

Eq.  3.5 

2x^  +  4x2  —  ^0 

Eq.  3.6 

X1  <  io 

Eq.  3.7 

xl>  x2  —  ® 

Eq.  3.8 

As  shown  in  Figure  12,  the  constraint  set  can  be  seen  to  define  a 
region  of  feasible  solutions.  (An  infeasible  problem  is  one  in  which 
all  constraints  cannot  be  satisfied  simultaneously.)  Any  point  within 
this  region  identifies  a  solution  which  satisfies  the  constraint  set. 
The  objective  is  then  to  define  that  point  (or  points)  which  maximizes 
the  value  of  the  objective  function.  Clearly,  this  will  occur  at  the 
boundary  of  the  region  of  feasible  solutions.  Graphically,  this 
determination  may  be  facilitated  by  first  plotting  a  line  representing 
the  slope  of  the  objective  function  which  passes  through  the  feasible 
region.  By  choosing  different  values  for  the  objective  function,  a 
family  of  lines  may  be  plotted  (Figurr  13) .  Each  of  these  isolines 
(Dannenbring  and  Starr,  1981)  will  be  parallel  to  the  others,  varying 
only  in  its  distance  from  the  origin.  The  optimal  solution  may  be 


Figure  12.  Linear  Programming  Constraint  Set 
(Adapted  from  Dannenbring  &  Starr,  1981) 


Figure  13.  Objective  Function  Isolines 
(Adapted  from  Dannenbring  &  Starr,  1981) 
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obtained  by  finding  the  objective  function  value  which  produces  the 
isoline  farthest  from  the  origin  while  still  touching  the  feasible 
region  (Figure  13)  .  The  values  of  the  decision  variables  are  then 
determined  by  the  point  at  which  that  intersection  occurs. 

A  variety  of  efficient  algorithmic  L.P.  solution  techniques  have 
been  developed.  Perhaps  the  best  known  of  these  is  the  simplex, 
devised  by  Dantzig  (1963).  Other  efficient  methods  utilize  the  dual 
formulation  of  the  L.P.  problem.  We  will  not  describe  these 
techniques  since  the  method  of  their  solution  is  not  relevant  to  the 
direction  of  this  research.  What  is  relevant  is  that  these  methods 
produce  provably  optimal  solutions  subject  to  the  limitations  of  data 
accuracy  and  linearity  of  the  objective  function  and  constraining 
equations . 

The  L.P.  approach  might  be  applied  to  the  design  decision  problem 
in.  at  least  two  ways  depending  on  how  we  define  the  decision 
variables.  The  decision  we  ultimately  desire  is  the  determination  of 
the  optimal  candidate  system.  Since  a  candidate  system  is 
distinguished  by  its  set  of  parameter  values  {y^l  it  may  be  wise  to 
use  the  parameters  as  decision  variables.  Following  this  approach, 
the  regional  and  functional  constraints  would  define  the  feasible 
region.  (Note,  however,  that  the  functional  constraints  may  produce 
discontinuities.)  Using  constant  relative  weights,  the  objective 
function  would  then  effectively  consist  of  the  criterion  function  with 
all  the  submodels  embedded.  Notationally ,  we  would  have: 
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max  CF  -  Ea^x^ 

Eq.  3.9 

subject  to 

Ecfciyk  £  d]_  for  all  1 
k 

Eq.  3.10 

where 

Xi  -  f*{Zj } ,  and 

Eq.  3.11 

Zj  -  gj{yk) 

Eq.  3.12 

(An  equivalent  alternative  would  be  to  define  the  submodels  as 
constraints . ) 

Unfortunately,  this  formulation  would  require  that  all 
constraints  and  submodels  be  represented  as  linear  combinations:  a 

restriction  not  inherent  in  the  design  methodology  itself  and  seldom 
true  in  complex  systems.  In  addition,  the  resulting  solution  would 
specify  some  combination  of  parameter  values  which  may  or  may  not 
represent  a  realizable  candidate  system.  It  might  be  possible  in 
simple  cases  to  add  constraints  to  increase  the  likelihood  of 
realizability.  However,  the  ability  to  successfully  define  such 
constraints  as  a  general  rule  is  speculative  and  the  additional 
solution  complexity  could  easily  become  prohibitive. 

An  alternative  formulation  could  be  conceived  in  which  the 
criterion  values  serve  directly  as  decision  variables: 

max  CF  -  Ea^x^ 

^cilxi  -  ^1  f°r  1 
i 


subject  to 


Eq.  3.13 
Eq.  3.14 
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In  this  formulation,  all  the  functional  and  regional  constraints 
would  have  to  be  directly  translated  to  linear  combinations  of  the 
(x^)  .  This  translation  could  well  be  impossible.  Even  in  the  case 
where  such  a  translation  is  possible,  the  optimal  values  of  the 
decision  variables  {x^}  would  still  have  to  be  "un- translated"  back 
into  some  combination  of  parameters  ly^} .  This  too  might  prove 
infeasible  or  difficult  at  best.  Finally,  even  if  such  a  conversion 
is  achieved,  the  combination  of  (y^)  may  not  represent  a  realizable 
candidate  system. 


Goal  Programming 

L.P.  is  generally  considered  to  be  a  single  objective  approach 
since  it  seeks  optimization  of  a  single  objective  function.  Even 
though  our  design  decision  problem  is  inherently  multi -objective ,  we 
were  able  to  consider  application  of  L.P.  by  using  constant  relative 
weights  { )  to  construct  a  single  linear  combination  of  the  multiple 
decision  criteria.  An  unfortunate  consequence  of  this  simultaneous 
consideration  was  the  computational  complexity  introduced  by  the 
interaction  of  submodels  due  to  shared  parameters .  An  alternative  to 
L.P.  which  was  devised  for  the  express  purpose  of  considering  multiple 
objectives  is  goal  programming  (Charnes  and  Cooper,  1960;  Ignizio, 
1976;  Lee,  1972). 


An  extension  of  the  basic  approach  to  L.P.,  goal  programming 
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recognizes  and  attempts  to  satisfy  a  rank- ordered  set  of  goals  or 
objectives.  If  it  is  unable  to  simultaneously  achieve  all  objectives, 
it  attempts  to  minimize  the  undesirable  deviation  from  objectives, 
insuring  higher  order  objectives  are  satisfied  first  (Dannenbring  and 
Starr,  1981).  It  may  also  be  formulated  and  solved  without  respect  to 
any  rank  ordering  of  goals,  striving  instead  to  minimize  the  total 
simultaneous  deviation  from  all  goals  (Hillier  and  Lieberman,  1980) . 

The  problem  formulation  is  similar  to  that  of  L.P.  Technological 
constraints  are  formed  as  before  to  deal  with  resource  limitations  and 
other  restrictions.  In  addition,  each  goal  is  converted  to  a  goal 
constraint  (Dannenbring  and  Starr,  1981)  which  includes  a  slack  or 
surplus  variable  which  represents  deviation  from  the  desired  level  of 
goal  achievement.  The  objective  is  then  stated  as  the  minimization  of 
the  (undesired)  deviations  in  the  goal  constraints .  In  the  case  where 
goals  are  rank  ordered,  the  objective  would  be  of  the  general  form: 

Minimize  D j '  |  Minimum  D^+  Eq.  3.15 

and  is  read  as ,  "minimize  Dj  *  given  the  minimum  D^+" ,  where  Dj  *  and 
D^+  are  undesirable  deviations  from  the  jth  and  ith  goals  and  where 
goal  i  has  higher  priority  than  goal  j  (Dannenbring  and  Starr,  1981). 

In  the  case  where  all  goals  are  considered  of  equal  importance 
(priority),  the  objective  would  be  stated  as: 
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Maximize  z  -  min{z^,Z2 . z^} 


Eq.  3.16 


which  is  read  as,  "maximize  the  minimum  progress  toward  all 
objectives"  (Hillier  and  Lieberman,  1980).  Differing  in  form  from  the 
first  case,  this  problem  would  be  formulated  as: 


subj  ect  to 


max  Z  -  z 


n 

2c j kXj -  z  >  0  f or  k  -  1 , 2 , . . . , K 
xj  >  0  for  j  -  1,2 . n 


Eq.  3.17 

Eq.  3.18 
Eq.  3.19 


In  the  latter  case,  solution  is  accomplished  using  standard  L.P. 
solution  techniques.  In  the  former,  solution  may  be  achieved  with 
only  slight  modification  of  the  simplex  method  (Dannenbring  and  Starr, 
1981)  . 


With  respect  to  the  design  decision  environment,  goal  programming 
provides  an  alternative  with  respect  to  the  treatment  of  multiple 
objectives.  Specifically,  it  permits  optimization  of  design  criteria 
in  an  ordinal  manner,  eliminating  the  necessity  for  assigning  relative 
weights.  In  cases  where  determination  of  relative  weights  is  an 
insurmountable  problem,  this  is  a  clear  advantage. 

However,  as  is  the  case  with  L.P.,  goal  programming  does  not 
provide  a  means  for  treating  varying  relative  weights.  Furthermore,  a 
price  is  incurred  with  its  use.  The  tradeoff  is  that  a  coarser 
distinction  exists  among  criteria  since  each  higher  priority  is 


completely  satisfied  (to  the  extent  possible)  before  the  next  lower 
priority  criterion  is  considered  at  all. 
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Another  characteristic  common  to  L.P.  is  that  if  interaction  is 
expected  or  observed  among  decision  criteria,  the  interaction  would 
likely  be  included  in  the  model  as  an  additional  constraint. 
Unfortunately,  by  adding  such  a  cons  .  .t,  the  decision  maker  is 
essentially  restricting  the  design  space  in  order  to  mandate 
interaction  rather  that  simply  responding  to  its  presence.  Therefore, 
superior  solutions  which  might  deviate  from  the  "normal"  pattern  of 
interaction  will  not  be  considered.  Furthermore,  since  the 
interaction  is  modeled  as  a  constraint  rather  than  being  included  in 
the  objective  function,  the  decision  maker  is  unable  to  express  the 
relative  importance  attached  to  joint  criteria  effects  in  the 
selection  process. 

Additionally,  the  method  requires  specification  of  target  levels 
of  achievement  which  have  a  direct  bearing  on  the  degree  to  which 
objectives  may  be  achieved.  In  the  uncertain  environment  of  design, 
these  targets  would  be  highly  speculative  aiid  might  easily  serve  to 
screen  possible  solutions  whose  ability  to  achieve  substantial 
improvements  in  lower  level  objectives  would  more  than  offset  a  small 
improvement  on  a  higher  level  objective.  The  specification  of  such 
targets  can  reasonably  be  described  as  encouraging  satisficing  rather 
than  optimizing  behavior  (Dannenbring  and  Starr,  1981). 


Finally,  goal  programming  makes  no  improvement  over  L.P.  in 
reducing  the  complexity  of  relating  criteria  values  to  specific 
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combinations  of  parameter  values .  Neither  does  it  offer  any 
additional  guarantees  that  the  solution  method  obtained  will  represent 
a  realizable  candidate  system.  While  it  is  generally  true  that  data 
characteristics  may  be  obtained  from  a  real  (or  potentially  real) 
system,  it  is  not  necessarily  true  that  a  real  system  may  be  achieved 
from  any  set  of  data  values.  In  that  sense  it  is  fair  to  say  that 
neither  L.P.  nor  G.P.  directly  evaluate  specific  candidate  systems. 
Instead,  they  seek  to  identify  the  theoretically  best  data  values  that 
could  be  obtained  under  the  stated  conditions. 

Decision  Analysis 

Decision  analysis,  also  referred  to  as  utility  theory,  has 
evolved  as  an  alternative  approach  to  multi -criteria  decision  making 
which  strives  to  closely  mirror  the  tradeoff  preferences  of  the 
decision  maker  while  retaining  a  prescriptive  perspective.  Stemming 
from  early  work  on  game  theory  by  von  Neumann  and  Morgenstem  (1944)  , 
the  technique  has  produced  a  large  body  of  literature  (Bell,  Keeney, 
and  Raiffa,  1977;  Cochrane  and  Zeleney,  1973;  Farquhar,  1977,  1980; 
Fishbum,  1964,  1970,  1972b,  1989;  Greenwood,  1969;  Howard,  1980, 
1988;  Huber,  1974a;  Keeney,  1978,  1982;  LaSalle,  1978;  Lifson,  1972; 
MacCrimmon,  1973;  Raiffa,  1968;  Starr  and  Zeleney,  1977)  which 
combines  quantitative  rigor  with  a  distinct  goal  of  practical 
application  on  large  and  complex  problems.  Recent  noteworthy 


developments  include  Saaty's  (1977,  1980,  1982,  1988)  Analytic 
Hierarchy  Process  approach  to  structuring  objectives  and  Harvey's 
(1986,  1988)  approach  to  the  consideration  of  long  term  consequences. 
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The  paradigm  of  decision  analysis  (Keeney  and  Raiffa,  1976b)  is 
shown  in  Table  5  and  is  described  as  follows.  In  the  Preanalysis 
step,  a  unitary  decision  maker  confronts  a  defined  problem  with  a 
specified  set  of  alternative  actions.  In  the  Structural  Analysis 
step,  the  decision  maker  constructs  a  decision  tree  such  as  the  simple 
example  in  Figure  14.  The  tree  is  composed  of  the  alternative  actions 
available  to  the  decision  maker  and  the  possible  outcomes  or 
consequences  associated  with  each  action.  During  Uncertainty 
Analysis,  the  decision  maker  assigns  a  probability  to  each  possible 
outcome.  (Note  here  that  Keeney  and  Raiffa,  like  most  decision 
analyst  researchers,  use  the  term  "uncertainty"  somewhat  differently 
from  the  way  it  was  used  by  Eilon  (1985) .  Whereas  Eilon  used 
"uncertainty"  to  simply  indicate  incomplete  information,  Keeney  and 
Raiffa  use  the  term  to  indicate  the  possibility  of  many  different 
outcomes,  each  with  some  probability  of  occurrence.  This  definition 
corresponds  most  closely  to  Eilon's  "decision  making  under  risk."  For 
the  remainder  of  this  section,  the  term  "uncertainty"  is  used  in  the 
sense  intended  by  Keeney  and  Raiffa.)  Next,  the  decision  maker 
undertakes  a  Utility  or  Value  Analysis.  This  consists  of  indicating 
his  or  her  preferences  by  assigning  a  utility  value  to  each  possible 
consequence.  In  the  case  of  uncertainty  (i.e.  multiple  consequences 
for  each  action,  each  with  an  associated  probability) ,  the  expected 


TABLE  5. 


PARADIGM  OF  DECISION  ANALYSIS 


Preanalysis 


Structural  Analysis 


Uncertainty  Analysis 


Utility  or  Value  Analysis 


Optimization  Analysis 
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utility  of  any  action  is  computed  as: 

m 

Sp^u^  Eq.  3.20 

i-1 

where  p^  and  u^  are  the  probability  and  the  utility  associated  with 
the  ith  consequence.  Finally,  in  Optimization  Analysis,  the  decision 
maker  selects  that  strategy  of  actions  which  maximizes  expected 
utility.  In  cases  where  a  series  of  actions  is  required,  dynamic 
programming  is  suggested  as  a  means  of  determining  the  optimal 
strategy. 

When  consequences  can  be  described  in  terms  of  a  single  attribute 
(criterion) ,  the  determination  of  the  optimal  decision  is  a  relatively 
straightforward  matter  as  described  above.  However,  in  the  case  of 
multi -attribute  decisions,  some  means  of  assessing  joint  preference 
over  all  attributes  simultaneously  must  be  found.  This  desired  value 
function  Is  denoted  as: 


v(x1,x2 . Xn)  -  f  [vj^Cxi)  ,v2(x2) . v^x^)]  Eq.  3-21 

where  v^  represents  a  value  function  over  the  single  attribute  X^. 
The  bulk  of  decision  analysis  is  concerned  with  the  search  for  and/or 
the  verification  of  this  function  which  is  said  to  define  the  decision 


maker's  preference  structure. 
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At  this  point,  it  is  appropriate  to  distinguish  the  terms  value 
function  and  utility  function.  Value  functions  apply  when 
consequences  are  assumed  to  be  known  with  certainty,  i.e.  the 
deterministic  case  (Keeney  and  Raiffa,  1976b:  68).  When,  as  described 
earlier,  uncertainty  prevails  so  that  consequences  have  associated 
probabilities,  i.e.  the  stochastic  case,  the  applicable  measure  of 
preference  is  termed  a  utility  function  (Keeney  and  Raiffa,  1976b: 
16) .  Although  an  extension  of  this  work  could  be  made  to  the 
stochastic  case,  we  have  so  far  in  the  design  context  not  defined 
uncertain  states  of  nature  or  their  associated  probabilities. 
Therefore,  unless  otherwise  specified,  we  will  confine  our  discussion 
to  the  use  of  value  functions . 

One  decision  alternative  is  said  to  dominate  another  when  the 
first  alternative  is  preferred  to  the  second  with  respect  to  each  and 
every  attribute  considered  independently.  Visualized  graphically  in 
the  simple  case  of  two  dimensions  (Figure  15) ,  we  may  plot  a  region  of 
possible  consequences  (related  to  possible  action  alternatives)  as 
points  represented  by  their  raw  values  for  each  of  the  two  attributes. 
All  points  (consequences)  not  dominated  by  any  other  point  are 
collectively  referred  to  as  the  efficient  frontier  or  Pareto  optimal 
set  (Keeney  and  Raiffa,  1976).  Clearly,  only  points  on  the  efficient 
frontier  should  be  considered  for  implementation  and  the  optimal 
strategy  must  lie  on  the  efficient  frontier. 


While  the  efficient  frontier  addresses  the  relative  performance 
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of  the  decision  alternatives,  it  provides  no  information  about  the 
preferences  of  the  decision  maker.  Toward  this  end,  points  in  the 
decision  space  considered  of  equal  preference  may  be  connected  to  form 
what  are  referred  to  as  indifference  curves  (Figure  16) .  Any  two 
points  on  the  same  curve  are  equally  preferable.  A  set  of  preference 
curves  is  collectively  described  as  a  preference  structure.  It  can 
then  be  seen  that  the  optimal  choice  is  the  point  farthest  from  the 
origin  at  which  the  efficient  frontier  intersects  the  decision  maker's 
preference  structure  (Figure  16) .  Restated,  the  optimal  choice  is 
that  alternative  which  has  the  greatest  preference  value  associated 
with  it.  Consequently,  the  decision  analyst  endeavors,  through 
interaction  with  the  decision  maker,  to  establish  the  value  functions 
which  define  the  preference  structure.  An  alternative  to  the  mapping 
of  the  complete  preference  surface  Is  the  hierarchical  structuring  of 
objectives  and  alternatives  to  permit  the  specification  of  preferences 
in  ratio  form  via  pairwise  comparisons  (Saaty,  1977,  1980,  1982,  1988; 
Olenik  and  Haimes,  1979). 

The  concept  of  independence  is  crucially  Important  in  the 
definition  of  value  functions.  Attribute  X^  Is  said  to  be 
preferentially  independent  of  X2  if  the  preference  structure  in  the  Xj 
space  does  not  depend  on  the  value  of  X2 .  The  presence  of  mutual 
preferential  independence  among  the  decision  criteria  means  that  the 
decision  maker's  preference  structure  may  be  specified  by  determining 
the  value  functions  for  each  of  decision  attribute  considered 
independently  of  the  other  attributes.  Under  this  condition,  an 


Figure  16 .  Indifference  Curves 
(Keeney  &  Raiffa,  1976b) 
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additive  value  function  is  defined  as 

v(x1,x2,  .  .  .5^)  -  ^(Xi)  Eq.  3.22 

If  the  preference  structure  of  X^  does  depend  on  the  value  of  X2, 
then  interaction  is  present  and  the  determination  of  the  preference 
structure  becomes  far  more  complex.  In  this  case,  extensive 
additional  discussion  must  ensue  between  decision  maker  and  decision 
analyst.  Rather  than  being  able  to  simply  define  marginal  (single¬ 
value)  attribute  preferences,  it  may  be  necessary  to  attempt  to  define 
the  complete  multi -dimensional  preference  structure  surface  by 
painstaking  comparison  of  individual  alternatives.  Not  surprisingly, 
much  effort  has  been  devoted  to  the  achievement  and  verification  of 
the  preference  function  or  of  preferential  independence  (Bell,  1979; 
Briskin,  1966,  1973;  Dewispelare  and  Sage,  1979;  Dyer  and  Sarin,  1979; 
Fishburn,  1972a;  Keeney,  1972,  1973a,  1976a;  Kirkwood  and  Sarin,  1980; 
Taylor,  1973).  Significant  additional  research  has  been  devoted  to 
practical  issues  such  as  sensitivity  analysis  (Barron  and  Schmidt, 
1988;  Evans,  1984)  and  applications  (Bard  and  Feinberg,  1989; 
Crawford,  Huntzinger,  and  Kirkwood,  1978;  de  Neufville  and  Keeney, 
1974;  Fishburn,  1967;  Huber,  1974a;  Keeney,  1973b,  1988;  Mehrez  and 
Sinuany-Stem,  1983). 

In  the  context  of  the  design  methodology,  the  value  or  utility 
associated  with  each  attribute  for  a  candidate  system  is  most 
accurately  described  as  the  product  a^X^  since  it  simultaneously 
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considers  both  the  quantity  of  the  attribute  possessed  and  the 
decision  maker's  relative  weight.  Consequently,  it  becomes  difficult 
to  differentiate  whether  the  specified  value  of  an  alternative  is 
predominantly  influenced  by  the  objective  performance  of  the  candidate 
system  or  the  decision  maker's  preference.  For  example,  take  the  case 
of  two  candidate  systems  and  their  performance  with  respect  to  one 
attribute.  Assume  candidate  system  #1  has  an  observed  value  of  333  on 
the  attribute  of  interest  and  candidate  system  #2  has  a  value  of  344. 
Further  assume  that  the  decision  maker's  value  function  assigns  values 
of  .2  and  .8  respectively  to  the  two  candidates  on  a  scale  of  0  to 
1.0.  Is  the  magnitude  of  the  difference  between  the  two  values 
primarily  an  indication  that  344  is  a  substantially  greater  attribute 
score  than  333  in  an  absolute  (or  objective)  sense  or  is  it  simply  a 
reflection  of  a  rapid  shift  in  the  relative  importance  of  the 
attribute  as  perceived  by  the  decision  maker?  Certainly  it  could  be 
argued  that  the  end  effect  is  the  same  regardless  of  the  reasons  which 
led  to  it.  However,  this  failure  to  separately  consider  the  objective 
performance  (X^)  and  the  subjective  input  (a^)  contributes  to  the 
difficulty  in  achieving  a  consistent  and  meaningful  preference 
structure.  Clearly,  the  difficulty  resulting  from  failing  to 
distinguish  these  two  factors  is  compounded  in  the  case  where  it  is 
necessary  to  simultaneously  consider  multiple  interacting  attributes 
(criteria) . 
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Dimensionless  Analysis 

The  final  general  quantitative  approach  we  review  is  that  of 
dimensionless  (or  dimensional)  analysis  (Bridgeman,  1922). 
Dimensionless  analysis  is  a  simple,  essentially  heuristic  technique 
which  is  expressly  concerned  with  comparing  alternatives  in  terms  of 
multiple  decision  criteria  expressed  in  noncomparable  units. 

Given  a  pair  of  alternatives ,  the  technique  eliminates  criteria 
units  by  forming  the  ratio  of  alternative  A's  performance  (in  raw 
units)  to  alternative  B's  on  each  criterion.  The  only  requirement  for 
criterion  measurements  is  that  the  direction  of  preference  is  common 
across  criteria,  i.e.  high  raw  criterion  scores  are  always  good  or 
always  bad.  If  we  assume  that  low  scores  are  preferred,  then 
alternative  A  is  preferable  to  B  for  a  given  criterion  if  the  ratio  of 
their  raw  scores  is  less  than  1.0.  If  the  ratio  is  greater  than  1.0, 
B  is  preferred. 

Each  criterion  is  assigned  a  relative  weight  on  the  scale  1  to  10 
and  the  performance  ration  for  each  criterion  is  raised  to  the  power 
of  the  relative  weight.  The  resulting  weighted  ratios  are  then 
multiplied  across  all  criteria  to  produce  a  single  number. 
Notationally ,  this  may  be  shown  as 

n 

j^Aj/Bi) 

i-1 


Eq.  3.23 
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where  and  B^  are  the  raw  performance  scores  for  alternatives  A  and 
B  on  the  ith  criterion  and  where  is  the  relative  weight  of  that 
criterion.  As  before,  if  the  overall  result  is  less  (greater)  than 
1.0,  A  (B)  is  preferred.  Clearly,  the  higher  the  relative  weight  of  a 
given  criterion,  the  greater  the  role  that  performance  ratio  plays  in 
the  overall  decision.  An  example  is  shown  in  Table  6. 

While  the  method  explicitly  considers  only  a  pair  of  alternatives 
at  a  time,  it  might  perhaps  be  extended  to  consider  multiple 
alternatives.  Alternately,  a  simple  tournament -style  pairing  approach 
can  be  used  in  which  the  winner  of  one  pairing  can  be  "played  off" 
against  the  winner  of  another  pair. 

Easy  to  use  and  logical,  the  method  must  nonetheless  be 
considered  heuristic  since  the  results  will  be  affected  by  the  initial 
scale  of  raw  criterion  measurements .  For  example ,  if  temperature  was 
a  criterion,  the  performance  ratio  and,  therefore,  the  results 
(although  not  necessarily  the  ultimate  decision)  would  be  affected  by 
whether  temperature  was  measured  in  Celsius  or  Fahrenheit.  In 
addition,  the  method  lacks  an  established  theoretical  basis. 
Therefore,  it  is  debatable  whether  independence  among  criteria  is 
implicitly  assumed. 


TABLE  6. 


DIMENSIONLESS  ANALYSIS 


Criteria 

Raw 

Scores 

A  B 

Ratio 

Weight 

Weighted 

Ratio 

Weight 

37 

39 

.949 

1 

.9490 

Cost 

6725 

4550 

1.480 

3 

3 . 2400 

Volume 

113 

142 

.796 

1 

.7960 

Safety  (inverted) 

2 

5 

.400 

4 

.0256 

Speed  (inverted) 

175 

75 

2.330 

2 

5.4400 

RESULT  0 . 3410 

(A  is  preferred) 
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Design-Based  Approaches 

The  quantitative  approaches  just  discussed  are  general  purpose  in 
nature  and  have  proven  their  worth  in  innumerable  applications. 
However,  it  is  frequently  the  case  that  a  general  purpose  tool,  while 
useful  in  a  variety  of  scenarios,  may  not  be  as  useful  in  any  specific 
case  as  a  tool  developed  especially  for  that  purpose.  For  this 
reason,  decision  making  techniques  have  been  developed  expressly  for 
use  in  the  macro-level  system  design  context.  In  the  following 
paragraphs  we  review  two  such  approaches:  the  Tricotyledon  theory  of 
system  design  and  the  criterion  function. 

Tricotyledon  Theory  of  System  Design 

Developed  by  Wymore  (1976) ,  this  approach  to  the  design  decision 
is  deeply  rooted  in  systems  theory.  It  pursues  the  use  of  set  theory 
and  mathematics  as  a  common  basis  for  combining  theories  from  a 
variety  of  disciplines  within  the  context  of  design  (Wymore,  1967). 

The  approach  begins  with  a  problem  statement  in  the  form  of  a 
target  system  definition.  Based  on  the  system  definition,  an 
input/output  specification  is  developed  which  attempts  to  define  all 
possible  combinations  of  inputs  over  time  (input  trajectories). 
Similarly,  the  plan  defines  all  possible  output  trajectories  and 
attempts  to  match  each  input  to  a  possible  set  of  outputs.  Next,  a 
set  of  systems  is  defined,  each  of  which  satisfies  the  input/output 
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specification  over  some  subset  of  time.  In  other  words,  for  any 
defined  input,  the  system  will  produce  one  of  the  matched  outputs  for 
some  period  of  time.  This  set  of  systems  is  referred  to  as  the 
input/output  cotyledon. 

A  separate  examination  of  technology  is  then  conducted  in  order 
to  identify  all  systems  that  can  be  assumed  to  be  available  as 
components  within  the  time  span  of  the  design  process.  These 
components  are  then  combined  via  coupling  recipes  to  form  potential 
systems.  The  resulting  set  of  systems  is  termed  the  technology 
cotyledon. 

The  feasibility  cotyledon  is  then  defined  as  the  intersection  of 
the  input/output  and  technology  cotyledons  (Figure  17)  .  Each  system 
in  the  feasibility  cotyledon  satisfies  the  input/output  specification 
over  some  period  of  time  and  can  be  achieved  from  a  coupling  recipe 
using  components  from  the  technology.  This  set  is  effectively  the  set 
of  candidate  systems. 

A  merit  ordering  process  is  then  used  to  select  the  optimal 
candidate  system.  The  process  suggested  by  Wymore  (1976)  for  this 
purpose  is  essentially  hierarchical  (Figure  18).  First,  experiments 
are  constructed  to  measure  characteristics  of  interest.  For  systems 
in  the  input/output  cotyledon,  these  characteristics  are  typically 
benefits,  while  the  characteristics  of  interest  in  the  technology 
cotyledon  are  usually  costs.  A  performance  index  is  assigned  as  the 


Input/Output  Technology 

Cotyledon  Cotyledon 


Figure  17.  Tricotyledon  Theory  of  System  Design 
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Figure  18 .  Merit  Ordering 


raw  performance  score  of  a  system  on  one  experiment.  Expected  values 
are  then  obtained  by  weighting  and  combining  the  performance  indices 
for  all  experiments  on  a  system.  Figures  of  merit  consist  of  expected 
values  of  performance  indices.  Finally,  an  order  of  merit  is  obtained 
by  weighting  and  combining  figures  of  merit.  This  merit  ordering 
process  is  first  conducted  independently  over  the  input/output  and 
technology  cotyledons.  A  final  merit  order  is  defined  over  the 
feasibility  cotyledon  in  such  a  way  that  it  is  consistent  with  the 
first  two  merit  orderings  while  specifying  tradeoffs  between  the  two 
to  permit  an  overall  ranking  of  candidate  systems . 

If  implemented  rigorously,  the  Tricotyledon  approach  can  clearly 
be  a  resource- intensive  process.  While  its  broad  systems  perspective 
increases  the  chances  of  considering  the  theoretic  optimum,  the 
magnitude  of  defining  all  possible  inputs  and  outputs  over  time 
represents  an  impressive  task  of  thoroughness.  In  addition,  the 
difficulty  of  identifying  the  intersection  between  the  input/output 
and  technology  cotyledons  cannot  be  overemphasized. 

Subjectivity  and  the  inclusion  of  relative  weights  are  permitted 
to  enter  the  merit  ordering  process  in  a  variety  of  ways.  Therefore, 
the  potential  exists  for  inconsistency  or  redundancy  in  the  weighting 
process.  However,  the  flexibility  of  the  approach  permits  the  exact 
structure  of  the  quantitative  evaluation  process  to  be  tailored  to  the 
application.  The  effectiveness  of  the  process  is  therefore  difficult 
to  generalize,  depending  largely  on  the  skills  of  the  designer. 
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Criterion  Function 

Resulting  from  original  work  by  Asimow  (1962)  and  extensions  by 
Ostrofsky  (1968,  1977d)  the  criterion  function  approach  to  the  design 
decision  problem  was  described  in  general  terms  in  Chapter  2.  In  the 
remaining  paragraphs  of  this  chapter  we  provide  a  more  detailed 
description  comparable  to  that  presented  for  the  other  decision 
techniques . 

As  described  earlier,  design  criteria  are  generally  not  specific 
enough  be  directly  measurable.  Consequently,  each  criterion  is 
logically  broken  down  into  constituent  elements.  Each  element,  in 
turn,  is  likewise  analyzed  until  a  set  of  measurable  characteristics 
(parameters)  is  identified  which  is  sufficient  to  represent  the 
criterion.  Those  criterion  elements  which  serve  as  intermediate  links 
between  criteria  and  parameters  are  referred  to  as  submodels. 
Quantitatively,  then, 


zj  “  sj(yk)  Eq.  3-24 

xt  -  fi(Zj)  Eq.  3.25 

and  xt  -  fi(gj(yk))  Eq-  3-26 


where  yjj  represents  the  kth  parameter,  zj  represents  the  jth  submodel, 
and  x^  represents  the  ith  criterion. 
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The  ultimate  combination  of  the  various  criteria  measures  into  a 
single  figure  of  merit  requires  conversion  to  unitless,  consistently 
scaled  values.  The  conversion  method  described  earlier  is  simple 
linear  normalization  of  the  form: 

X  ■*  (x  -  ^in)/ (^ax  "  ^in^  ®*q.  3.27 

However,  to  avoid  the  assumption  of  linearity  of  importance  and  to 
provide  a  means  for  the  treatment  of  criteria  interaction,  the 
preferred  method  is  the  use  of  probability  theory.  Specifically,  the 
raw  criterion  scores  of  all  candidate  systems  are  combined  to  form  a 
cumulative  distribution  function  (c.d.f.)  for  each  criterion  (Figure 
19) .  Each  value  of  xj[  is  then  converted  to  some  value  of  on  the 
range  0  to  1  using  the  appropriate  c.d.f. 

It  may  be  useful  at  this  point  to  pause  and  consider  the  use  of 
probability  theory  in  this  line  of  research.  To  many  readers,  the 
term  "probability"  invokes  images  of  random  events ,  repeated 
observation  of  outcomes,  and  the  quantitative  description  of  the 
relationships  between  them.  However,  as  Savage  (1954),  Anscombe  and 
Aumann  (1963),  and  many  others  have  pointed  out,  probability  has 
equally  Important  meaning  and  applicability  when  used  in  a  much 
broader,  logical,  subjective  sense.  Concerned  primarily  with  personal 
preferences,  as  opposed  to  the  strict  concept  of  chance,  this  latter 
meaning  serves  as  the  philosophical  and  quantitative  basis  for  the 
application  of  probability  in  both  utility  theory  and  the  criterion 
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function  approach  described  in  this  paper.  It  is  imperative  to 
remember  that  our  objective  is  to  use  the  theory  as  an  objective  tool 
to  help  insure  an  optimal  (Keen,  1977)  choice  can  be  achieved,  given 


the  inherently  subjective  nature  of  the  environment. 

To  this  point,  the  discussion  has  focused  exclusively  on  the 
observable  or  objective  characteristics  of  the  system  being  planned. 
However,  in  most  cases  of  complex  systems,  it  is  unlikely  that  all 
design  criteria  are  considered  to  be  of  equal  importance.  Although  it 
may  be  reasonable  to  include  such  subjective  considerations,  it  is 
desirable  to  treat  these  considerations  as  objectively  as  possible. 
Consequently,  each  criterion  is  assigned  a  relative  weight  (a^) . 
Although  any  of  a  number  of  methods  may  be  used  to  obtain  these 
relative  weights  (Huber,  1974b;  Choo  and  Wedley,  1985;  Eckenrode, 
1965;  Gershon,  1984;  Nutt,  1980),  they  are  normalized  in  such  a  way 
that: 


Sat  -  1.0  Eq.  3.28 

and  probability  theory  may,  again,  be  used  as  a  guide  in  subsequent 
manipulations . 

In  many  cases,  it  is  unreasonable  to  expect  that  relative  weights 
will  remain  the  same  for  all  possible  criterion  values.  For  example, 
consider  the  simplistic  case  where  only  two  criteria,  cost  (x^)  and 
operational  availability  (X2),  are  being  considered.  Assume  that  a 
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project  budget  has  been  established  at  the  midpoint  of  the  range  of 
project  costs  considered  plausible.  Up  to  the  point  where  the  budget 
is  exceeded,  maximizing  availability  may  be  far  more  important  than 
the  money  required  to  achieve  it.  Once  the  budget  is  exceeded, 
however,  the  difficulty  of  obtaining  additional  funds  suddenly  becomes 
substantially  more  difficult.  Beyond  that  point,  the  cost  criterion 
may  be  considered  far  more  important  (relatively)  than  the  higher 
availability  obtained.  This  situation  is  illustrated  in  Figure  20 
where  higher  values  of  indicate  better  performance  on  each 
criterion.  (Therefore,  the  budget  is  exceeded  at  values  of  X^  below 
the  mid-point.) 

The  first  four  classes  of  models  (I-IV)  in  Figure  21  have  been 
differentiated  based  on  the  behavior  of  the  relative  weights  over  the 
range  of  their  respective  criteria.  In  Model  I,  the  relative  weights 
for  all  criteria  are  constants.  In  Model  II,  the  relative  weights  of 
all  criteria  are  constant  within  specified  intervals;  however,  the 
relative  weight  of  at  least  two  criteria  change  at  least  once 
throughout  the  X^  range.  Model  III  may  be  thought  of  as  an  extension 
of  Model  II  to  the  case  where  an  infinite  number  of  intervals  exist 
such  that  the  relative  weights  are  approximated  by  continuous 
functions  which  vary  throughout  their  ranges.  Model  IV  may  be 
considered  a  combination  of  Models  II  and  III. 

In  order  to  obtain  a  single  figure  of  merit  for  each  candidate 
system,  the  criteria  values  and  their  respective  relative  weights  are 
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Figure  20.  Varying  Relative  Weights 
(Simons  &  Ostrofsky,  1988) 
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combined  in  the  form  of  a  criterion  function  (CF)  .  The  form  most 
frequently  used  for  this  purpose  has  been  the  additive  form,  which 
sums  all  criterion  values,  with  each  multiplied  by  its  relative 
weight: 


CF  -  SCaiXXi)  Eq.  3.29 

The  value  of  CF  is  computed  for  each  candidate  system.  The 
candidate  with  the  highest  value  of  CF  is  considered  the  optimal 
candidate  system.  While  the  theoretical  optimal  system  would  have  CF 
-  1.0,  there  is  no  assurance  that  such  a  value  will  be  obtained  since 
the  theoretical  optimal  combir  .u.^n  of  parameter  values  may  not 
represent  a  realizable  candidate  system.  (In  applications  to  date,  no 
such  candidate  system  has  emerged.)  However,  a  search  of  the  design 
hyperspace  for  such  a  theoretical  candidate  is  still  useful,  since  a 
better  candidate  may  have  been  overlooked.  Even  if  not,  the  location 
of  higher  values  of  CF  may  represent  desirable  directions  for  future 
improvement . 

Unfortunately,  the  strictly  additive  form  of  the  criterion 
function  may  not  always  be  appropriate.  As  mentioned  earlier, 
multiple  criteria  frequently  have  interactions  among  them.  Clearly, 
our  previous  example  was  based  on  a  direct  interaction  between  cost 
and  availability.  It  is  largely  for  the  purpose  of  dealing  with  such 
interactions  that  probability  was  chosen  as  the  preferred  method  for 
generating  normalized  X^. 
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The  first  step  in  determining  criteria  interaction  (X^j )  is  to 
verify  its  existence.  This  may  be  done  visually  by  plotting  the 
values  of  each  candidate  system  on  a  graph  whose  axes  correspond  to 
the  interacting  criteria.  The  resulting  pattern  (the  functional 
interaction)  may  then  be  approximated  as  a  continuous  function  (e.g. 
via  regression)  as  illustrated  in  Figure  22.  Statistical  validity  of 
the  approximated  function  may  be  supported  using  a  variety  of 
nonparametric  tests  (e.g.  Kolmogorov- Smirnov) .  Based  on  current 
practice,  this  function  (f(x^,Xj))  is  then  converted  to  a  c.d.f.  of 
the  form:  F(x^,Xj|xj).  Using  Bayes's  Theorem,  the  interaction  term 
(Xjj )  may  then  be  computed  by  multiplying  the  conditional  probability 
from  the  c.d.f.  times  the  marginal  probability  (i.e.  X^j  - 
F(xi,xj |xj )F(xj ) ) . 

Since  criteria  interactions  (Xjj)  are,  of  themselves,  events 
(Ostrofsky,  1977d:  361),  they  are  assigned  relative  weights  (a^j)  in 
the  same  manner  as  the  marginal  criteria  (Ostrofsky,  1977d:  374).  The 
requirement  still  exists  that  the  sum  of  all  relative  weights  equals 
1.0. 


Models  I -IV  assume  no  interaction  among  criteria.  Therefore,  the 
inclusion  of  interaction  terms  results  in  four  additional  classes  of 
models  (V-VIII)  as  shown  in  Figure  21.  (Note  that  our  research 
objective  concerns  Model  VII.)  Treating  the  interaction  terms  based 
on  probability  theory,  we  have: 
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CF  —  2a  jX^  +  (a^jX^j) 

+  2Sijk(aijkXijk> 

+  ^ijk.  .  .n(aijk.  .  .n^ijk.  .  .n)  ^q.  3.30 

(-l)s,  when  interaction  exists  and  where  s  is  the 
order  of  interaction  when  interaction 
exists  (e.g.  X^j  is  first  order 
interaction,  X^k  is  second  order, 
etc. ) 

0,  when  no  interaction  exists 

Since  this  approach  endeavors  to  use  the  laws  of  probability  as  a 
guide,  the  resulting  value  of  CF  should  always  fall  between  0  and  1.0 
for  each  candidate  system.  Unfortunately ,  the  current  criterion 
function  implementation  technique  may  permit  CF  values  less  than  0. 
This  indicates  that,  in  some  manner,  the  process  used  to  apply 
probability  theory  has  been  in  error. 


As  our  description  above  suggests,  perhaps  the  most  difficult 
step  in  the  design/planning  process  is  the  actual  selection  of  a 
specific  candidate  system  which  maximizes  the  achievement  of  the 
specified  criteria.  Inputs  to  the  decision  process  are  both  objective 
and  subjective.  Objective  inputs  include  the  definition  of  the 
candidate  systems,  their  respective  subsystems,  most  parameter  values, 
and  quantifiable  goals.  Subjective  inputs  include  tradeoffs  among 
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conflicting  objectives  and  personal  preferences.  The  primary  desired 
output  of  the  decision  process  is  the  identification  of  the  "best" 
candidate  system,  where  "best"  refers  to  the  relative  best  from  among 
the  alternatives  considered  (although  this  may  or  may  not  be  the 
theoretical  "ideal").  In  addition,  it  is  frequently  desirable  to 
determine  a  ranking  of  all  candidate  systems  as  well  as  the  relative 
difference  among  them. 

The  literature  contains  a  wealth  of  approaches  (Blin,  1977; 
Bridgeman,  1922;  Cheng  and  Mclnnis,  1980;  Cohen  et  al,  1985;  Epstein, 
1957;  Green  and  Srinivasan,  1978;  Harrison,  1975;  Holloway,  1979; 
Ignizio,  1976;  Janis  and  Mann,  1977;  Keeney,  1982;  Miller,  1970;  Roy, 
1977;  Rubinstein,  1975,  1980,  1986;  Sage,  1977;  Sakawa  and  Seo,  1980; 
Shull,  Delbacq,  and  Cummings,  1970;  Simon,  1960,  1978;  Tribus,  1969; 
White  and  Sage,  1980;  Wymore,  1976;  Yager,  1981;  Zeleney,  1973,  1974, 
1976a,  1976b,  1977,  1980,  1981a,  1981b,  1982,  1984)  which  attempt  to 
model  (either  prescriptively  or  descriptively)  the  human  decision 
making  process  in  such  a  way  that  inputs  similar  to  those  defined 
above  are  converted  to  some  form  of  useful  output.  Noteworthy 
approaches  which  have  been  widely  cited  and/or  used  include  linear 
programming,  goal  programming,  decision  analysis,  dimensionless 
analysis,  and  design-based  techniques.  However,  each  of  these  methods 
has  demonstrated  deficiencies  which  are  summarized  in  Table  7  and  in 
the  paragraphs  which  follow. 


While  linear  programming  and  goal  programming  permit 
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COMPARISON  OF  DECISION  MAKING  TECHNIQUES 
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consideration  of  multiple  objectives,  neither  method  accommodates 
varying  relative  weights  and  both  restrict  relations  among  variables 
to  linear  combinations  and  do  not  explicitly  evaluate  candidate 
systems.  Even  more  importantly,  both  methods  produce  a  theoretically 
optimal  solution  resulting  from  a  search  of  the  feasible  region  of 
solutions.  However,  there  is  no  assurance  that  this  solution 
corresponds  to  a  realizable  candidate  system. 

The  perceived  potential  benefits  of  decision  analysis  have  not 
been  fully  realized  in  practice  (Howard,  1988) ,  perhaps  because  of  a 
logical  contradiction  between  the  theory's  intent  to  descriptively 
model  human  preferences  and  its  axiomatic  assumption  of  rationality 
(Edwards  and  Luce,  1988).  Of  more  interest  in  our  search  for  a 
prescriptive  decision  making  approach  is  the  failure  of  decision 
analysis  to  separate  objective  considerations  (i.e.  IX^))  from 
subjective  ones  (i.e.  (a^)).  Another  shortcoming  of  utility  theory 
techniques  is  difficulty  in  translating  measures  of  merit  to  tangible 
or  measurable  characteristics  of  the  candidate  systems.  This 
difficulty  frequently  undermines  the  confidence  of  users  in  the 
ultimate  decision.  It  is  not  easy,  for  example,  to  convince  a  company 
vice  president  that  product  design  A  is  preferred  to  B  because  it  has 
.037  more  utiles  than  B.  Finally,  failure  to  establish  the  required 
independence  may  necessitate  a  painstaking  point-by-point  definition 
of  the  decision  maker's  complete  preference  structure. 

While  dimensionless  analysis  distinguishes  objective  from 
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subjective  considerations  and  permits  direct  relation  to  observable 
system  characteristics,  it  does  not  support  varying  relative  weights 
and  its  results  must  be  considered  heuristic. 

Wymore's  Tricotyledon  approach  permits  a  great  deal  of 
flexibility:  perhaps  too  much  so.  In  addition  to  being  potentially 
cumbersome  to  apply,  there  is  very  little  restriction  of  the  manner  in 
which  subjective  and  objective  considerations  are  combined  during 
merit  ordering,  making  the  optimality  of  its  solutions  arguable. 

Finally,  most  of  these  techniques  fail  to  account  explicitly  for 
interactions  among  design  criteria.  Instead,  they  may  require 
extensive  measures  to  test  for  independence,  require  modification  of 
the  problem  so  that  no  interaction  is  observed,  or  simply  ignore 
interactions  altogether.  (Although  linear  and  goal  programming  permit 
specification  of  interaction  as  additional  constraints,  the 
interaction  must  be  capable  of  being  represented  in  a  linear  form. 
Furthermore,  this  approach  would  force  the  optimal  solution  to  conform 
to  the  prevailing  observed  interaction,  even  in  cases  where  exceptions 
might  be  possible.)  The  primary  exception  to  this  generalization  is 
the  criterion  function.  Unfortunately,  some  of  the  past 
implementations  of  this  approach  have  obtained  results  which  are  not 
uniformly  consistent  with  the  underlying  theory. 
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Specific  Research  Goal 

One  could  pursue  the  broad  research  objective  stated  in  Chapter  1 
by  attempting  to  resolve  weaknesses  in  any  of  the  techniques 
discussed.  However,  since  the  criterion  function  approach  contains 
all  the  desired  characteristics  and  is  directly  linked  to  the  design 
framework,  we  perceive  the  most  promising  alternative  to  be  the 
identification  and  resolution  of  the  discrepancies  which  have  produced 
the  observed  anomalies  in  its  results.  Resolution  of  these 
discrepancies  would  enable  the  criterion  function  to  achieve  the 
research  objective  stated  in  Chapter  1.  Therefore,  while  the  overall 
research  objective  remains  as  stated,  a  key  goal  to  the  success  of 

clarification  of  the  criterion  function 


this  research  will  be  the 


Chapter  4 


CRITERION  FUNCTION  DEVELOPMENT 

As  a  preliminary  note,  it  should  be  remembered  that  the  criterion 
function  uses  probability,  not  in  the  context  of  uncertainty,  but 
rather  as  a  theoretical  convention  for  quantifying  performance  in  such 
a  manner  that  we  are  able  to:  1)  compare  candidate  systems  across 
criteria,  and  2)  treat  interactions  amonp  criteria. 

Parzen  (1960:  29-30)  divides  the  literature  of  probability  theory 
into  three  broad  categories:  1)  the  nature  or  foundations  of 
probability,  2)  mathematical  probability  theory,  and  3)  applied 
probability  theory.  He  describes  mathematical  probability  theory 
literature  as  consisting  of  those  writings  which  are  primarily 
axiomatic.  By  contrast,  the  literature  of  applied  probability 
consists  of  writings  in  which  probability  serves  as  a  tool  in 
scientific  or  scholarly  investigations.  Since  Ostrofsky's  (1968) 
origination  of  the  use  of  probability  as  a  basis  for  the  criterion 
function  was  a  novel  application,  his  work  was  necessarily  concerned 
with  establishing  a  strong  linkage  between  the  axiomatic  world  of 
probability  mathematics  and  the  criterion  function  application.  By 
contrast,  the  current  research  is  able  to  build  on  the  formal  linkage 
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previously  established,  focusing  more  exclusively  on  its 

implementation  in  the  criterion  function. 

Current  Method  of  Criterion  Function  Synthesis 

Prior  to  recommending  changes  to  the  current  decision  technique, 
it  is  necessary  to  revisit  the  underlying  theory  which  serves  as  the 
basis  for  the  method  and  the  ways  in  which  it  is  currently  applied. 
This  is  necessary  for  two  reasons.  First,  the  rationale  for  the 
current  method  must  be  understood  if  we  are  to  be  able  to  identify 
where  its  methods  of  implementation  have  failed.  Second,  our  goal  is 
to  more  correctly  apply  the  under lying  theory  of  the  current  approach. 
Therefore,  the  theoretical  basis  for  the  present  approach  also  serves 
as  the  basi-,  for  the  approach  proposed  in  this  research:  the 

difference  lies  in  the  clarification  of  that  theory. 

Criterion  Measurement 

The  obtainment  of  raw  criterion  values  for  each  candidate  system 
through  the  parameters  and  submodels  was  described  in  Chapters  2  and 
3.  It  was  also  stated  that  some  method  of  conversion  is  necessary  to 
remove  the  distortions  of  the  raw  measurement  scales  in  order  to 
permit  comparison  across  criteria.  Folkeson  (1982)  referred  to  this 
as  the  dimensionality  problem. 


Two  methods  of  accomplishing  this  compatibility  of  scale  were 
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described:  linear  normalization  based  on  the  minimum  and  maximum 
values  and  use  of  probability  theory.  Unfortunately,  the  linear 
normalization  approach  suffers  from  the  influence  of  the  original 
measurement  scale  in  a  manner  similar  to  that  experienced  by  the 
dimensionless  analysis  approach.  Specifically,  the  results  produced 
are  directly  influenced  by  the  initial  scale  of  measurement  and  by  the 
definitions  of  minimums  and  maximums.  On  the  other  hand,  probability 
theory  provides  a  vehicle  for  comparison  which  is  strictly  relative  in 
nature,  i.e.  the  performance  of  a  candidate  system  is  judged  solely  in 
terms  of  its  relation  to  that  of  other  candidate  systems,  regardless 
of  the  initial  scale  of  measure.  In  addition,  probability  theory  was 
selected  by  Ostrofsky  (1968)  for  use  in  the  criterion  function  largely 
because  it  is  the  only  branch  of  mathematics  which  provides  an 
explicit  means  of  treating  interactions.  The  following  paragraphs 
provide  a  synopsis  of  the  theoretical  basis  for  the  use  of  probability 
theory  in  the  criterion  function.  To  permit  more  detailed  reference, 
the  most  immediately  relevant  portions  of  Ostrofsky' s  work  (1977d) 
have  been  included  as  Appendix  A. 

The  Applicability  of  Probability  Theory  to  Criterion  Measurement 

Essentially,  Ostrofsky  (1968,  1977d)  defined  the  existence  of  a 
candidate  system  as  an  event.  The  remainder  of  the  universe  of 
events,  then,  consists  of  all  possible  events  (candidate  systems) 
within  the  design  space.  A  function  can  then  be  defined  which  assigns 
to  the  occurrence  of  each  event  (candidate  system)  a  description 
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(measure)  of  that  candidate  with  respect  to  a  specific  design 
criterion.  If  a  set  function  is  then  defined  over  the  descriptions  in 
such  a  way  that  the  axioms  of  probability  theory  are  satisfied,  that 
set  function  qualifies  as  a  probability  measure. 

In  other  words,  Ostrofsky  recognized  and  capitalized  on  (as  has 
Wymore,  1976)  the  fact  that  probability  theory  is  a  set  of 
mathematical  descriptions  and  resulting  tools  which  can  be  applied  to 
virtually  any  phenomena  which  meet  the  required  definitions .  More 
explicitly,  the  use  of  probability  theory  is  entirely  appropriate  even 
in  cases  where  uncertainty,  hypothesis  testing,  or  statistical 
inference  are  not  central  issues.  The  only  requirement  is  that  the 
theory's  conventions  be  adhered  to. 

In  this  context,  then,  the  candidate  system  is  considered  an 
event,  the  performance  of  the  candidate  system  (for  a  given  criterion) 
is  defined  as  the  relevant  description  resulting  from  some  function 
(i.e.  criterion  modeling),  and  the  descriptions  produced  can  then  be 
acted  on  by  a  set  function  which  satisfies  probability  axioms. 

Implementing  Criterion  Measurement 

Based  on  the  rationale  just  presented,  each  candidate  system's 
value  of  the  raw  criterion  measure,  x^,  is  produced  as  a  function  of 
the  observed  event  within  the  range  [x^  min>  xi  maxi •  A  probability 
density  function  (p.d.f.)  may  then  be  defined  (as  the  necessary  set 


function)  which  acts  on  the  raw  criterion  scores ,  x^ ,  in  a  manner 


consistent  with  probability  theory.  A  cumulative  distribution 
function  (c.d.f.)  may  then  be  constructed  from  the  p.d.f.  However, 
since  our  interest  lies  in  the  cumulative  scores,  a  c.d.f.  is  normally 
constructed  directly  from  the  raw  criterion  scores  (Figure  23)  without 
explicit  representation  of  the  p.d.f.  The  c.d.f.  then  serves  as  the 
vehicle  for  conversion  of  the  raw  scores  to  a  unitless  value  (denoted 
X^)  on  the  range  [0,1].  Since  all  criteria  have  been  converted  in  the 
same  manner,  comparability  across  criteria  is  now  possible.  Note  that 
the  use  of  the  c.d.f.  eliminates  all  perspective  of  the  absolute 
quality  of  a  given  candidate's  performance  on  a  criterion.  Instead, 
all  measures  become  completely  relative,  serving  to  identify  only  how 
each  candidate  compares  with  the  remainder  of  the  candidate  system 
set . 


Interaction  Measurement 

The  most  substantial  work  concerning  interactions  within  the 
framework  of  the  criterion  function  has  been  that  of  Ostrofsky  (1968, 
1977d,  1987)  and  Folkeson  (1982).  Ostrofsky's  work  primarily 
highlighted  the  need  for  treatment  of  interactions,  suggested  the  use 
of  probability  theory  as  a  means  for  that  treatment,  and  described  an 
approach  for  measurement  of  interaction  among  the  criterion  (X^)  . 
Folkeson  (1982)  later  termed  this  type  of  interaction  functional  (as 
opposed  to  preferential)  interaction.  In  the  following  paragraphs,  we 
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Figure  23.  Criterion  Cumulative  Distribution  Function 
(Simons  &  Ostrofsky,  1988) 
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present  Ostrofsky's  (1977d,  1987)  approach  to  the  use  of  probability 
in  measuring  functional  interaction.  Since  this  material  is  crucially 
important  to  our  subsequent  discussion,  we  quote  directly  from 
Ostrofsky  (1977d:  361-362). 

(Begin  quote) 

Functional  Interactions  as  Probabilities 


Let 

-  the  event  (Xj<x^) 

Ajj  -  the  event  (X^<X£,Xj<Xj) 

Aijk...(J+l)  “  the  event  (Xi<*llXj<xj . XJ+1<xJ+1) 

where 


J  +  1  —  n 


Then 


SiPCAi)  - 


fiijP(Aij) 


marginal  distribution  function  of  the 
ith  criterion: 

1m  when  P(A^)  exists 

0,  when  P(A^)  does  not  exist 

-  joint  probability  or  the  first-order 
interaction  of  x^  and  xj : 

1,  when  P(A^j)  exists 

0,  when  P(A^j)  does  not  exist 
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*ijk.  .  .  (J+l)pIAijk.  .  .  (J+l)l  "  probability  or  the 

jth- order  interaction  of 

xi >xj . X(J+1)  • 


*ijk...(J+l) 


when  P[Ai;jk. ..(J+1)]  exists 

when  P[Aijk ...(j+i)]  does  not  exist 


where  the  x^s  are  admissable  values  of  the  X^. 


It  should  be  noted  that 

P(Ai)  -  F(Xi) 

P(Aij)  -  F(xt,xj)  Eq.  4.1 


ptAijk. . .(J+1)] 


Ftxi>xj . X(J+1)1 


The  concept  of  the  union  of  events  in  a  probability  space  has 
been  discussed  in  Appendix  A.  When  these  events  are  not  mutually 
exclusive  they  can  be  shown  as  (Mood  and  Graybill,  1963) 
n 

Z  -  P(  U  A±) 
i-1 


n 

2  «iP(Ai) 

i-1 


n  n 

2  2  6ijP(A£j ) 

W 


n  n  n 

+  2X2  «ijkp<Aijk) 
ijk  J  J 

j/k 

i/k 


Eq.  4.2 


n  n  n 


i  j 
Mj 


...26 

J+1 


ijk. . . (J+l)ptAijk. . .(J+1)] 


J/J+l 
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Hence  the  problem  of  identifying  additional  candidate  systems  can 
be  resolved  to  the  problem  of  identifying  a  particular  value  of  Z,  the 

union  of  all  events,  A^|i-1 . n.  The  degenerate  case  exists  where 

the  are  mutually  exclusive  and 

n  n 

Z  -  P(  U  -  2  PCAi)  Eq.  4.3 

i-1  i-1 


(End  quote) 

Measuring  (Computing)  Functional  Interactions 

Let  x-^  and  X2  be  two  design  criteria  with  a  density  function 
f(x^,X2).  From  laboratory  testing  or  other  means  the  points  along 
g(x^,X2)  can  be  obtained  (Figure  24a).  (Note  that  g(x]L,X2)  is  a  two- 
dimensional  projection  of  the  density  function,  f(x^,X2).)  These 
points  are  then  plotted  and  the  Kolmogorov- Smirnov  test  used  to  fit 
the  theoretic  distribution  F(x^,X2lx^)  (Figure  24b).  Based  on  Bayes's 
Theorem,  the  interaction  term  is  then  computed  as 

X12  -  F(xltx2)  -  F(x1,x2|x1)*F(x1)  Eq.  4.4 


For  the  second-order  interaction,  the  value  of  the  third 
criterion  resulting  from  the  combination  of  the  first  two  is  plotted 
as  shown  in  Figure  25a.  In  this  manner,  h(x^,X2,X3)  Is  transformed 


1.0 

F(x1,x2|x1) 

0 

X1  min  X1  max 

X1 

(b) 

Figure  24.  Quantitative  Description  of  Functional  Interaction 

a.  Determination  of  g(xi,X2) 

b.  Conversion  to  Cumulative  Distribution  Function 


(Ostrofsky,  1977d) 


g(xi,x2)max 


g(x1,x2) 


e(xi-x2> 


(a) 


(b) 

Figure  25.  Second  Order  Interaction 

a.  Determination  of  h(x^,x2,X3> 

b.  Conversion  to  Cumulative  Distribution  Function 


(Ostrofsky,  1977d) 


into  the  probability  space  as  shown  in  Figure  25b.  This  procedure  is 
continued  until  all  interactions  have  been  transformed  into 
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probabilities . 


Inclusion  of  Relative  Weights 

In  the  following  paragraphs,  we  review  the  three  major 
developments  in  the  inclusion  of  relative  weights:  the  basic  role  of 
relative  weights  within  the  context  of  probability  theory  (Ostrofsky, 
1968,  1977d) ,  the  possibility  and  treatment  of  varying  relative 

weights  (Ostrofsky,  1977d,  1987),  and  the  concept  of  preferential 

independence  (Folkeson,  1982) . 

Rejfttjve  Weights  ,»s  .fyobgblHUes 

In  the  previous  section  it  was  shown  that  the  union  of  events 
which  are  not  mutually  exclusive  results  in  Equation  4.2,  where  the 
probabilities  are  a  function  of  the  criteria  values  resulting  from  a 
given  candidate  system.  When  the  relative  weight,  a^,  of  each 
criterion  is  included,  the  result  will  be  a  real  number  which,  when 
obtained  for  each  candidate  in  the  set  of  candidate  systems  for  the 
design,  defines  the  hierarchy  of  the  desirability  of  the  candidates  by 
establishing  the  list  of  candidates  with  the  highest  values  at  the  top 
of  the  hierarchy. 

i- 


A  raw  measure  of  relative  value,  a' 


can  be  transformed  to  a 
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subjective  probability,  a^  (Fishbum,  1970).  Ostrofsky  observes  that 
when  -  P[X£<x^] ,  an  examination  of  the  a^  and  P(A^)  indicates  that, 
in  the  context  of  design,  the  a^  are  statistically  independent  of 
P(A^)  since  a^  will  remain  constant  for  any  value  of  P(A^)  within  the 
given  interval  of  x^ ;  hence 


P(Ai)  -  ajPCAi)  -  0L  Eq.  4.5 


Recall  that,  as  we  have  defined  it,  P(A^)-F(xj)  so  that 
0£“ajF(X£).  Then,  when  the  x^  are  not  mutually  exclusive,  Equation 
4.2  may  be  revised  as  follows, 


n 

CF  -  P(  U  a^) 
i-1 


n  n  n 

-  2  *  2  X 

i— 1  i  j  J  J 

i/j 

n  n  n 

+  X  X  X  ^Hk^ilk  Eq.  4.6 

i  j  k  J 

j?* 

i/k 

n  n  n 

"  1  i  j  "■J^15iJk---^J+1>(?iJk---<J+1) 

i/j 

J/J+l 

where 

6i#i  “  ^iaiP(Ai)  “  ^iaiF(x£>  Eq .  4.7 
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Equation  4.7  may  be  described  as  the  weighted  marginal  probability  of 
the  ith  criterion,  where 


Similarly, 


when  8 ^  exists 

when  does  not  exist 


5ij^ij  “  5ijaijp(Aij)  “  5ijaijF(xi*xj)  Etl-  48 


This  is  the  weighted  value  of  the  first-order  interaction  of  the  x^ 
and  xj  as  measured  by  their  joint  marginal  distribution  where 


1,  when  exists 
0,  when  8 jj  does  not  exist 

Likewise , 


5ijk*ijk  “  6ijkaijkp(Aijk>  “  5ijkaijkF(xi<xj -xk>  Ecl ■  4-9 

This  term  is  the  weighted  value  of  the  second-order  interaction  of  x^, 
x j ,  and  Xfc  as  measured  by  their  joint  marginal  distribution  where 


5ijk 


1,  when  exists 

0,  when  0^^  does  not  exist 


Therefore,  the  general  form  for  any  term  in  Equation  4.6  becomes 
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5ijk.  .  .  (J+l)*ijk. . .  (J+l) 

“  5ijk.  . .  (J+l)aijk.  . .  (J+l)ptAijk.  .  .  (J+l)  1 

-  6i1k. . . (J+l)ai1k. . . (J+l)Ffxi *xj >xk> • • • *X(J+1)] 

Eq .  4.10 

Equation  4.6  then  defines  the  relative  value  of  a  candidate 
system  to  include  both  the  relative  importance  of  the  respective 
criterion,  x^,  and  the  value  of  the  probability  density  of  the  x^. 

Varying  Relative  Weights 

It  is  entirely  reasonable  (Ostrofsky,  1977d:  372)  for  relative 
weights  to  assume  different  values  at  different  points  on  the  range 
(xi  min*  xi  max) •  In  ot^®r  words,  a  situation  may  exist  where  one  set 
of  {a^}  exists  for  {x^}  when 

xi  min^xi~x ' i *  i“l , . . . , n  Eq .  4.11 

and  another  set  of  {a'^)  exists  when 

x' jiSXjix' '  i-l,...,n  Eq.  4.12 

and  a  third  set  of  (a''^J  when 

x' '  jSxjSx"  '  £ ,  i-1 . n  Eq.  4.13 


and  so  on  until  there  is  a  value  of  a^  for  all  intervals  in  the 
complete  range  of  each  x^  (see  Table  8) .  It  was  this  possibility 
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TABLE  8. 

RELATIVE  WEIGHTS  VARYING  BY  INTERVAL 
(Ostrofsky,  1977d) 


Interval 


al 

a2 

a3 


% 


I 

II 

III 

e+1 

xi  min— xI^xi ' 

Xi'<Xi<Xi' ' 

Xj  ’  '  <Xi<X£ '  '  ' 

• • •  xif<xi— xi  max 

al' 

ax'  ' 

«!' ' ' 

ai£ 

a2' 

a2" 

a2' ' ' 

«4> 

CM 

cd 

a3' 

a3" 

a3' ' ' 

a3c 

an' 

an" 

an'" 

V 

n 

Sai'-l 
i  * 

n 

Sa£' '-1 
i 

n 

Ea^' ' '-1 
i 

.  n 

Sa^-l 

i 
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which  led  to  the  proposed  existence  of  the  eight  decision  models  shown 
in  Chapter  3  of  this  dissertation.  However,  Ostrofsky  observes 
(1977d:  372)  that,  "the  properties  of  a  probability  density  apply  to 
the  {a^}  for  each  predefined  interval  in  the  range  of  {x^}." 
Therefore , 

n 

Sa^  -  1  Eq.  4. 14 

i 


for  each  interval  although  the  a^  do  not  necessarily  -urn  to  1  over  the 
total  range  of  a  given  x^. 

When  more  than  one  interval  exists,  Ostirofsky  states  that  the 
relative  weights  must  be  compared  within  the  same  interval,  where 
transitivity  holds  across  criteria  (as  shown  in  Table  8) .  This  is 
intended  to  support  a  direct  comparison  of  the  desirability  of 
candidate  systems  by  comparing  their  resulting  real  numbers  in  [0,1]. 

Given  any  set  of  relative  weights  which  vary  across  intervals, 
Ostrofsky  (1987)  has  suggested  a  two-step  normalization  scheme  for 
computation  of  the  relative  weights  which  insures  Equation  4.14 
remains  satisfied.  This  scheme  consists  of  vertical  and  horizontal 
normalization  of  the  a^,  where  the  terms  vertical  and  horizontal  refer 
to  the  orientation  of  Table  8 . 

Vertical  normalization  means  that  the  relative  weight  for  the  kth 
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criterion  in  the  p  interval  (a^)  is  first  divided  by  the  sum  of  all  n 

in  the  p  interval.  When  a  finite  number  of  intervals  have  been 
defined,  this  normalization  can  be  accomplished  once  prior  to 
evaluating  candidate  systems  so  that  all  relative  weights  in  the  same 
interval  already  satisfy  Equation  4.14  as  in  Table  8.  Horizontal 
normalization  is  then  accomplished  by  insuring  that  Equation  4.14  is 
satisfied  for  each  candidate  system.  This  is  achieved  by  dividing 
each  a^  for  each  candidate  system  by  the  sum  of  the  a^  for  that 
candidate  system. 

Model  V  (constant  relative  weights  with  interactions)  was 
implemented  by  Peschke  (1986)  in  optimizing  an  information  resource 
management  system.  The  case  (just  described)  of  relative  weights 
varying  throughout  the  [0,1]  range  but  remaining  constant  within  each 
defined  interval  is  accommodated  by  Models  II  and  VI,  depending  on  the 
presence  of  interactions.  Model  VI  was  demonstrated  by  Folkeson  for 
the  design  of  a  piece  of  aircraft  support  equipment.  A  logical 
extension  of  this  case  is  the  scenario  in  which  relative  weights  vary 
continuously  throughout  the  range  (Model  VII) .  This  may  be  thought 
of  as  an  extension  of  Models  II  and  VI  in  which  there  are  an  infinite 
number  of  intervals.  This  model  has  been  suggested  (Folkeson,  1982; 
Peschke,  1986;  Ostrofsky,  1987)  but  not  previously  demonstrated. 

Preferential  Independence 


As  mentioned  earlier,  Folkeson  (1982)  addressed  two  forms  of 
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interaction:  functional  (interaction  among  the  X^)  and  preferential 
(interaction  among  the  a^)  .  Although  he  suggested  a  variety  of 
methods  for  identifying  potential  functional  interaction  (e.g. 
designer  suspicions,  correlation  coefficients,  regression,  ^tc.),  he 
provided  no  specific  suggestions  for  treating  those  interactions  and 
the  example  he  described  had  no  functional  interactions  which  he  was 
able  to  quantify.  However,  by  drawing  on  the  field  of  decision 
analysis,  Folkeson  directly  addressed  the  treatment  of  the  lack  of 
preferential  independence.  In  the  following  paragraphs,  we  paraphrase 
the  most  significant  portion  of  this  contribution. 

As  stated  during  the  discussion  of  decision  analysis  in  Chapter 
3,  the  presence  of  interaction  among  criteria  may  lead  to  the  need  for 
complete  multi -dimensional  specification  of  the  decision  maker's 
preference  structure.  According  to  Keeney  and  Raiffa  (1976),  the  pair 
of  attributes  X  and  Y  is  preferentially  independent  of  Z  if  the 
conditional  preferences  in  (X,Y)  space  given  Z'  do  not  depend  on  Z' . 

For  example, 

(Xlt  Y1(  Z)  >  (X2,  Y2,  Z),  for  all  Z 

Keeney  and  Raiffa  (1976:105)  state  that  the  additive  form  is 
appropriate  only  when  all  attributes  are  mutually  preferentially 
independent : 


134 

A  value  function  v  may  be  expressed  in  an  additive 

form 

v(X,  Y,  Z)  -  vx(X)  +  vy(Y)  +  v2(Z) 

where  vx,  Vy,  and  vz  are  single  attribute  value  functions, 
if  and  only  if  {X,  Y)  is  preferentially  independent  of  Z, 

{X,  Z)  is  preferentially  independent  of  Y,  and  {Y,  Z}  is 
preferentially  independent  of  X. 


Subsequent  work  by  Dyer  and  Sarin  (1979)  and  by  Dewispelare  and 
Sage  (1979)  showed  the  multiplicative  form  was  appropriate,  even  in 
the  presence  of  interaction,  as  long  as  the  attributes  are  mutually 
weak  difference  independent  (WD1).  Folkeson  (1982)  describes  WDI  as 
follows: 


Define  the  symbol  Xj  to  indicate  all  components  of  X  not 
contained  in  X^.  Then,  criterion  X^  is  WDI  of  Xj  if  given 
any  b^,  cj_,  d^,  e  X^,  such  that 

v(blt  bj)  -  v(cit  bj)  £  v(di(  bj)  -  v(eif  bj)  Eq.  4.15 

for  some  bj  e  Xj,  then  it  is  required  that 

v(bicj)  -  v(cj_C£)  >  v(d£Cj)  -  v(e^cj)  Eq.  4.16 

for  any  cj  e  Xj. 

As  the  number  of  criteria  present  in  a  given  decision 
problem  increase,  the  proof  of  WDI  can  become  very  tedious. 
The  verification  of  the  appropriateness  of  the 
multiplicative  form  of  the  criterion  function  requires 
checking  all  subsets  of  criteria  for  WDI.  Dewispelare  and 
Sage  (1979)  developed  a  theorem  to  make  this  task  simpler 
and  less  time-consuming. 

Theorem:  Given  criteria  X^,X2 . Xp,  the  following  are 

equivalent . 

a)  Criteria  X]_  ,X2 ,  .  .  .  ,Xn  are  mutually  weak  difference 
independent  (MWDI). 
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b)  is  weak  difference  independent  of  Xj,  and 
(X^.Xj),  j/i,  is  preferentially  independent  (PI); 
j-1,2,3, . . . ,n,  n>3 . 


Folkeson  indicates  that  preferential  independence  should  have 
been  checked  before  considering  the  additive  model  in  the  first  place. 
He  therefore  concludes  that  checking  for  WDI  in  X^  and  Xj  is  all  that 
is  required  to  proceed  in  using  the  following  multiplicative  form  of 
the  criterion  function: 

n  n 

CFX  —  2  a^X^  +  K  2  a^a^X^X-s 
i— 1  i— 1  J  J 

j>i 


n 

+  2  a^aj  ajjX^XjXjj 

j>i 

k>j 

n 

+  .  .  .  +  Kn_1  n  a^  Eq.  4.17 

i-1 


or 


n 

1  +  K(CFX)  -  n  [1  +  KajXi]  Eq.  4.18 

i-1 


He  further  observes  that  if  the  X^  are  preferentially  independent 
and  the  a^  sum  to  1.0,  the  additive  criterion  function  is  appropriate 
(i.e.  K  -  0.0).  However,  if  the  a^  do  not  sum  to  1.0,  the 
multiplicative  form  of  the  criterion  function  should  be  used  with  the 
scaling  constant  K  derived  from  the  a^  values  using  the  following 


relation 
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n 

1  +  K  -  it  (1  +  Ka^  Eq.  4.19 
i-1 

The  correct  value  of  K  is  obtained  through  convergence  by 
repeatedly  solving  Equation  4.19.  When  the  sum  of  the  a^  is  greater 
than  1.0,  K  will  fall  between  -1.0  and  0.0.  When  the  sum  of  the  a^  is 
less  than  1.0,  K  will  be  greater  than  0.  (See  Keeney  and  Raiffa, 
1976:307,324-325,347-348;  and  Sage,  1977:246-250.) 

Folkeson  suggests  that  sensitivity  analysis  should  again  be 
performed  at  this  stage  to  provide  the  decision  maker  with  not  only 
discrete  value  measures  but  also  a  feel  for  the  marginal  effects  in 
the  area  of  the  optimal  candidate  or  around  other  points  of  interest. 

Note  that  Folkeson  identified  the  lack  of  preferential 
independence  by  the  failure  to  satisfy  Equation  4.14.  Recall, 
however,  that  the  vertical  and  horizontal  normalization  scheme  of 
Ostrofsky  (1986)  will  insure  that  this  relationship  is  always 
satisfied.  Clearly,  then,  two  different  schools  of  thought  exist  in 
this  regard.  The  distinction  between  these  approaches  probably  lies 
in  their  fundamental  perception  of  the  significance  of  the  a^  and  in 
particular,  the  interaction  relative  weights  (e.g.  ajj ) .  We  perceive 
this  to  be  an  important  issue  which  should  be  pursued  in  future 
research.  However,  since  pursuit  of  this  distinction  is  beyond  the 
scope  of  the  present  research,  we  will  henceforth  employ  the  double 


normalization  scheme  of  Ostrofsky,  which  has  been  applied  in  practice 
to  date . 
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Revised  Approach  to  Criterion  Function  Synthesis 

Before  proceeding  with  a  detailed  re-evaluation  of  the  methods 
just  reviewed,  it  is  important  to  reexamine  the  intuitive  or  logical 
basis  for  the  form  of  the  criterion  function  itself.  In  so  doing,  we 
seek  to  avoid  confusion  concerning  the  sometimes  subtle  meaning  of 
apparently  obvious  concepts . 

By  way  of  justifying  this  discussion,  consider  Bertrand's  paradox 
(Parzen,  1960:  302).  The  paradox  begins  with  the  following  problem: 

Let  a  chord  be  chosen  randomly  in  a  circle  of  radius  r. 

What  is  the  probability  that  the  length  X  of  the  chord  will 

be  less  than  the  radius  r? 

This  problem  is  capable  of  producing  a  variety  of  different  answers, 
each  seemingly  quite  logical,  yet  varying  in  value  from  0.134  to 
0.333.  The  paradox  is  caused  by  a  lack  of  clarity  concerning  what  is 
meant  by  a  randomly  chosen  chord.  Each  of  the  different  answers  is 
either  correct  or  incorrect  depending  on  which  interpretation  of  the 
problem  is  accepted  at  the  outset.  From  this  paradox  we  conclude 
that,  even  for  seemingly  well-defined  events,  a  thorough  reexamination 
of  the  meaning  of  that  event  may  shed  significant  light  on  how  we 
compute  the  solution.  Therefore,  we  proceed  by  focusing  on  the 
intuitive  meaning  of  the  event (s)  of  interest. 
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Logical  Basis  for  the  Form  of  the  Criterion  Function 

Parzen  (1960:  12)  defines  an  event  as  a  set  of  descriptions.  "To 
say  that  an  event  E  has  occurred  is  to  say  that  the  outcome  of  the 
random  situation  under  consideration  has  a  description  that  is  a 
member  of  E."  In  the  context  of  design  decision  making,  each 
candidate  system  may  be  seen  as  an  event  whose  descriptions  are  stated 
in  terms  of  parameter  values  and  the  resulting  criteria  values . 

Define  as  the  event  that  the  candidate  system  being  evaluated 
is  preferred  to  another  candidate  system  with  respect  to  the  ith 
criterion.  Define  0 ±  as  the  probability  associated  with  event  e^, 
i.e.  0jL-P[€jJ.  (Note:  For  the  moment,  we  will  not  attempt  to 
separate  the  constituent  elements  of  performance,  X^,  and  relative 
importance,  A^.  By  stating  that  a  candidate  system  is  preferred,  we 
temporarily  accept  that  this  description  includes  consideration  of 
both  performance  and  relative  importance.  However,  we  will  make  a 
distinction  between  the  two  In  the  sections  following  this  more 
general  discussion.) 

In  simple  terms,  what  we  ideally  want  is  the  candidate  system 
which  has  the  greatest  value  of  0^  for  all  criteria  simultaneously. 
In  other  words ,  we  want  the  candidate  system  which  outperforms  all 
other  candidate  systems  on  each  and  every  criterion.  Such  a  candidate 
system,  if  it  exists,  would  be  said  to  dominate  the  other  candidate 
systems  (Zeleney,  1977).  In  the  case  where  such  a  candidate  system 
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exists,  the  decision  is  trivial  since  the  dominating  candidate  system 
is  clearly  superior  in  all  respects. 


The  challenge,  then,  is  how  to  decide  when  no  candidate  system 
exists  which  is  dominant  with  respect  to  all  criteria  simultaneously. 
One  way  to  make  such  a  decision  is  to  try  to  minimize  the  probability 
of  making  a  bad  choice.  In  our  context,  the  worst  possible  choice 
would  be  the  selection  of  a  candidate  system  which  is  inferior  on  all 
criteria  simultaneously.  In  terms  of  ej_  and  6±,  such  a  choice  would 
have  the  lowest  values  of  9 ^  relative  to  the  other  candidate  systems. 
Alternatively,  we  may  define  e^c  as  the  complement  of  :  the  event 
that  the  candidate  system  being  evaluated  is  not  preferred  to  another 
candidate  system  on  the  ith  criterion.  Similarly,  define  as  the 
probability  associated  with  ej®.  Then,  for  a  bad  choice,  values  of 

would  be  relatively  high  since  P[ e^c]-l-P[ ejJ  and,  therefore, 
df-l-ef  (Parzen,  1960:  19). 

Stated  in  this  manner,  our  goal  would  be  to  find  that  candidate 
system  which  has  the  smallest  values  of  9±c  for  all  i  simultaneously. 
However,  due  to  the  reciprocal  nature  of  complementary  events,  no  such 
candidate  system  exists  or  we  would  have  identified  it  initially  as 
dominating  all  other  candidate  systems.  Therefore,  a  reasonable 
alternative  goal  is  to  come  as  close  to  this  ideal  as  possible:  to 
seek  that  candidate  system  which  is  least  likely  to  be  a  bad  choice, 
i.e.  that  candidate  system  which  has  the  smallest  simultaneous  (across 
all  i)  combination  of  9^c. 
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According  to  Parzen  (1960:  13),  the  intersection  (EF)  of  two 
events  "is  defined  as  consisting  of  the  descriptions  that  belong  to 
both  E  and  F;  consequently,  the  event  EF  is  said  to  occur  if  and  only 
if  both  E  and  F  occur,  which  is  to  say  that  the  observed  outcome  has  a 
description  that  is  a  member  of  both  E  and  F."  The  simultaneous 
occurrence  of  the  is,  therefore,  defined  as  their  intersection 

since  the  collection  of  €£C  for  a  given  candidate  system  constitutes 
the  description  of  that  candidate  system  in  terms  of  each  i.  A 
candidate  system  is  characterized  by  simultaneous  descriptions  with 
respect  to  all  criteria.  Therefore,  we  are  interested  in  finding  that 
candidate  system  which  has  the  minimum  value  of  the  probability  of  the 
intersection  of  its  c^c. 

Parzen  (1960:  12)  would  denote  the  intersection  of  the  e^c  as 
eice2c...enc  where  n  is  the  number  of  criteria.  It  is  from  this 
representation  that  the  intuitive  basis  for  the  form  of  the  criterion 
function  emerges. 

According  to  de  Morgan's  laws  (Parzen,  1960:  16),  for  n  events, 
El>  e2> • • • >En» 

(El  U  E2  U...U  E n)c  -  E1cE2c...Enc  Eq.  4.20 

Therefore,  we  have  that 

£lC£2C---£nC  -  <£1  u  £2  u---u  £n>C  E9-  4-21 

In  other  words,  in  order  to  avoid  a  bad  choice  (when  no  candidate 
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system  dominates) ,  we  wish  to  minimize  the  probability  of  the  event 
(£1  U  e2  13.  .  .U  en)c,  i.e. 

Min  (#j.  U  $2  U.  .  .U  0n)c  Eq .  4.22 

Due  to  the  nature  of  complementary  events,  we  have  (Parzen,  1960:  19) 

Min  (6X  U  &2  U. . .U  0n)c  -->  Min  1-(0X  U  02  U. . .U  0n)  Eq.  4.23 

-->  Max  (0!  U  02  U...U  6n)  Eq.  4.24 

If  the  «£  are  mutually  exclusive,  then 
Max  U  62  U...U  Bn)  -  Max  ( 81  +  82  +...+  8n )  Eq.  4.25 
However,  for  two  events  to  be  mutually  exclusive,  one  event  must  be 
able  to  occur  without  the  occurrence  of  the  other.  Therefore,  in  the 
context  of  design  decision  making,  to  permit  mutually  exclusive  8 ^  is 
to  permit  the  evaluation  of  candidate  systems  which  do  not  have 
descriptions  (or  observed  behavior)  on  all  criteria.  While  this  may 
be  theoretically  possible,  historical  application  of  the  criterion 
function  has  been  restricted  to  candidate  systems  which  have 
descriptions  (measurements)  for  all  criteria.  In  other  words,  the 
are  not  mutually  exclusive.  On  the  contrary,  the  existence  of  a 
candidate  system  infers  the  presence  of  for  all  i.  (This 

assumption  is  clearly  embedded  in  Ostrofsky's  original  work  (1968, 
1977d)  as  well.)  We  consider  this  a  reasonable  assumption  and  accept 
it  for  the  balance  of  this  research.  In  the  worst  case,  the 
assumption  serves  only  to  limit,  but  not  invalidate,  our  conclusions. 

When  two  events  are  not  mutually  exclusive,  their  union  is 
computed  (Parzen,  1960:  19)  as 


P[E  U  F]  -  P[E]  +  P[F]  -  P[EF] 


Eq.  4.26 
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where  P[EF]  is  the  probability  associated  with  the  intersection  EF,  as 
defined  previously  (Parzen,  1960:  13).  For  the  case  of  two  criteria, 
then,  we  have  that 

P[«i  U  c 2 3  “  P[«il  +  P[«2l  *  pt€12]  Eq-  ^-27 

where  e^2  is  the  intersection  of  events  and  C2-  Equivalently, 

(ffj.  U  6 2 )  “  +  0 2  "  ^12  Eq .  4.28 

For  the  more  general  case  of  n  criteria,  Equation  4.28  is  extended  as 

n  n  n 
(0X  U  02  U...U  0n)  -  S01  -  S  s 

i-1  i-1  j-i+1 

+ 

n  n 

±  E  ...  S  m  Eq.  4.29 

i-1  m-1+1 

where  all  6  with  more  than  one  subscript  represent  the  probabilities 
of  the  intersections  of  the  associated  events . 

Note  that  the  form  of  the  relationship  just  derived  is  precisely 
that  of  the  criterion  function  (Eq.  4.6).  Our  goal  is  to  find  the 
candidate  system  which  maximizes  that  expression.  In  so  doing,  the 
preceding  discussion  reveals  that  we  are  using  the  relationships  of 
probability  theory  to  make  the  best  possible  decision:  that  which 

minimizes  the  probability  of  being  inferior. 

The  sections  which  follow  extend  the  reexamination  begun  here  by 
analyzing  the  effects  of  this  discussion  on  the  constituent  elements 
of  the  6:  the  X^,  the  A^,  and  the  intersection  terms  These  sections 
will  identify  some  inconsistencies  between  the  current  method  of 


criterion  function  implementation  and  the  underlying  theory.  The 
inconsistencies  are  a  reflection  of  the  limitations  of  the  current 


methods  of  implementation  and  are  not  a  condemnation  of  the  criterion 
function  approach  in  general  or  the  work  of  previous  researchers. 

The  intent  of  the  proposed  research  is  to  suggest  and  demonstrate 
revisions  to  the  current  methods  of  implementation  which  will  resolve 
the  inconsistencies.  These  revisions  will  be  based  on  the  rediscovery 
of  simple  truths.  The  rediscovery  process  will  require  that  we  back 
up  and  correctly  apply  what  had  previously  been  discovered  to  be  true 
but  has  somehow  been  overlooked  in  the  implementation  process  over 
time,  and  will  in  itself  demonstrate,  not  only  increased  knowledge  and 
awareness  of  the  process,  but  also  effective  research. 

Reexamination  of  Criterion  Measurements 

In  the  following  sections  we  focus  our  general  discussion  of  the 
8^  to  the  specific  subcategory  of  the  X^. 

Linear  Normalization  vs.  Probability 

The  first  elementary  issue  in  reexamining  the  synthesis  of  the 
criterion  function  is  a  reexamination  of  the  basic  approach  to 
characterizing  candidate  system  performance  with  respect  to  a  given 
design  criterion.  Given  a  set  of  raw  (observed)  criterion  performance 
measurements,  two  methods  of  conversion  to  permit  unitless  comparison 


have  been  suggested  thus  far:  linear  normalization  using  x^  m^n  and 
xi  max  an<*  conversion  to  probabilities  via  cumulative  distribution 
frequencies  (c.d.f.).  Let  us  briefly  reconsider  the  strengths  and 
weaknesses  of  the  two  approaches. 

The  primary  distinguishing  feature  of  linear  normalization  is 
that  the  resulting  measures  attempt  to  reflect  an  absolute  value  of 
performance.  By  this  we  mean  that  the  normalized  value,  X^,  reflects 
the  performance  of  a  candidate  system  solely  with  respect  to  the 
constant  range  [x^  min'  xi  max^ *  If  a  candidate  system's  raw 
performance  exceeds  50%  of  the  defined  criterion  range,  its  score  will 
be  .5,  regardless  of  how  well  the  other  candidate  systems  perform. 
This  has  the  advantage  of  reflecting  dramatic  differences  in  the 
performance  of  candidate  systems  in  terms  of  an  absolute  benchmark 
scale . 

By  contrast,  the  c.d.f.  approach  to  criterion  scoring  is 
completely  relative.  Regardless  of  where  a  candidate's  raw  score 
falls  in  the  [x^  min,  x^  max]  range,  its  criterion  performance  will  be 
computed  strictly  as  a  function  of  how  that  candidate  compared  to  all 
others  observed.  Therefore,  even  minute  changes  in  the  absolute 
performance  of  a  candidate  may  cause  a  dramatic  change  in  Xj_  if  the 
candidates  are  closely  grouped. 

However,  a  significant  drawback  of  the  normalization  approach  is 
its  strict  dependence  on  the  definitions  of  x^  m^n  and  x^  max.  Given 
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that  comparison  across  criteria  will  ultimately  serve  as  the  basis  for 
selection,  the  determination  of  the  possible  ranges  of  the  criteria 
may  clearly  influence  the  decision.  Take,  for  example,  the  trivial 
case  of  two  candidate  systems  and  two  equally  important  criteria  as 
shown  in  Table  9 . 

When  X2  max  changes  from  10  to  100,  the  overall  decision  changes 
from  favoring  candidate  A  to  candidate  B.  While  this  characteristic 
may  be  desirable  where  clearly  defined  and  meaningful  minimums  and 
maximums  exist,  it  is  an  undesirable  trait  in  the  more  common  case 
where  the  ranges  are  speculative  or  arbitrary.  By  contrast,  the 
probability  based  approach  is  completely  insensitive  to  such  range 
changes . 

Clearly  then,  the  use  of  probability  provides  a  better  means  for 
assessing  relative  performance,  which  is,  in  fact,  the  crux  of  the 
design  decision.  Recall  that  the  design  decision  is  concerned  with 
the  identification  of  the  candidate  system  which  is  relatively  best 
from  among  the  set  considered.  Note  that  the  determination  of  how 
much  better  one  candidate  is  than  another  will  be  determined  by  the 
shape  of  the  c.d.f.  curve.  As  described  so  far,  the  shape  of  the 
c.d.f.  will  be  determined  by  the  performance  of  the  defined  set  of 
candidate  systems.  However,  if  one  is  inclined  to  assume  that  the 
performance  distribution  of  the  defined  set  is  representative  of  the 
distribution  of  the  theoretic  exhaustive  set,  then  the  relative 
performance  of  a  candidate  system  is  effectively  assessed  relative  to 
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TABLE  9. 

EXAMPLE  CANDIDATE  SYSTEM  CF  SCORES 


Candidate  ^  x2 
A  4  9 

B  5  1 


CF  based  on  x^.xo  ranges 
xl“ [0,10]  xx- [0,10] 

X2=ULlfil  x2-T 0.1001 


,  65 


.30 


.245 


.  255’ 


where  CF  -  .5XX  +  .5X2 


*  Preferred  candidate 


all  possible  candidate  systems  including  those  which  are,  as  yet, 
undefined. 
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The  implications  of  assumptions  concerning  the  shape  of  the 
c.d.f.  can  be  carried  a  step  further.  Assume  for  a  moment  that  we  do 
not  consider  the  performance  distribution  of  the  defined  set  of 
candidates  to  be  representative  of  the  theoretic  exhaustive  set  but 
that  we  have  some  reason  to  suspect  some  other,  specific  distribution. 
In  this  case,  it  seems  entirely  appropriate  to  utilize  the  suspected 
theoretical  distribution  to  construct  the  c.d.f.  and  produce  the  X*. 

In  summary  then,  we  have  observed  that  unlike  the  linear 
normalization  approach,  the  use  of  probability  provides  a  strictly 
relative  measure  which  is  insensitive  to  arbitrary  definitions  of  the 
raw  criteria  ranges.  In  addition,  we  have  said  that  the  relationships 
observed  among  candidates  using  the  probability  based  approach  may  be 
generalized  to  unspecified  candidates  if  we  consider  the  observed 
distribution  representative  of  a  broader  case.  On  the  other  hand,  we 
suggest  that  known  or  suspected  distribution  characteristics  of  the 
complete  (theoretical)  set  of  alternatives  may  shed  new  light  on  those 
candidates  we  have  defined.  Finally,  we  recall  that  probability 
provides  a  theoretic  basis  for  treating  interactions. 

Probability  as  a  Measure  of  Dominance 


For  the  reasons  described  above  e  accept  the  use  of  probability 
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theory  as  the  preferred  basis  for  comparison  across  criteria.  As 
shown  earlier  in  this  chapter,  Ostrofsky  (1968,  1977d)  has  rigorously 
established  the  application  of  that  theory  in  the  context  of  the 
design  decision  criteria.  With  respect  to  measurement  of  marginal 
criteria  values ,  we  indorse  that  contribution  and  provide  the 
following  amplifying  comments. 

Recall  that  the  probability  based  measure  of  the  performance  of  a 
candidate  system  with  respect  to  a  given  criterion  is  defined  as  F(x) 
-  P{7:X(y)<x) .  Verbally,  this  may  be  stated  as  follows:  the 

performance  of  a  candidate  system  with  respect  to  a  given  criterion  is 
the  probability  that  the  raw  performance  of  the  candidate  being 
evaluated  exceeds  that  of  another  candidate  system  on  the  criterion  of 
interest.  Those  candidates  whose  performance  is  exceeded  may  be 
described  as  being  dominated  with  respect  to  the  ith  criterion  by  the 
system  being  evaluated.  Therefore,  X^  effectively  measures  the 
portion  of  the  set  of  candidate  systems  which  is  dominated  on  the  ith 
criterion  bv  the  candidate  system  being  evaluated  (Keeney  and  Raiffa, 
1976;  Zeleney,  1977).  Graphically,  this  is  shown  in  Figure  26. 

Measuring  Intersections  and  Interaction 

Interaction  vs.  Intersection 

If,  as  we  have  asserted  in  the  previous  section,  the  significance 
and  computation  of  the  X^  are  appropriate  and  correct,  then  the 
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Portion  of  set  of  candidate 
systems  dominated  by  a 
on  ith  criterion 


Xia 


Figure  26.  Single  Criterion  Dominance 
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source  of  the  unexplained  results  (i.e.  CF<0)  must  lie  in  the 
computation  of  the  interaction  terms,  the  relative  weights,  or  both. 
Indeed,  we  will  suggest  that  this  is  precisely  the  case.  In  this 
section,  we  consider  what  Folkeson  (1982)  termed  functional 
interactions ,  namely  the  X^j .  That  aspect  of  interaction 
characterized  by  Folkeson  as  preferential  interaction  will  be 
discussed  in  the  subsequent  section  on  the  inclusion  of  relative 
weights . 

To  begin  our  discussion,  we  recall  our  earlier  examination  of  the 
form  of  the  criterion  function  as  expressed  in  Equation  4.6.  Clearly, 
this  equation  is  the  generic  representation  of  the  computation  of  the 
probability  of  the  union  of  the  n  marginal  events  (Ostrofsky,  1977: 
362).  Consequently,  we  see  that  the  terms  heretofore  referred  to  as 
interaction  terms  are  more  accurately  the  probabilities  of  the 
intersections  of  the  various  marginal  events.  Visually,  the  Venn 
diagram  (e.g.  Figure  27)  provides  a  useful  means  for  recognizing  the 
unions  and  intersections  of  events .  The  intersection  of  events  A  and 
B  is  the  event  that  both  A  and  B  occur,  sometimes  referred  to  as  the 
joint  occurrence  of  A  and  B  (Ross,  1988). 

Focusing  now  specifically  on  the  X^,  recall  that  we  have  defined 
a  marginal  criterion  measure  as  F(x)  -  P(7:X(7)<x).  We  have  verbally 
described  a  marginal  criterion  measure  as  the  probability  that  the  raw 
performance  of  the  candidate  being  evaluated  exceeds  that  of  any  other 
candidate  system  on  the  single  criterion  of  interest.  Alternately,  we 
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27 .  Venn  Diagram  of  Intersecting  Criteria 


have  described  it  as  the  portion  of  the  set  of  candidate  systems  which 
is  dominated  on  the  ith  criterion  by  the  candidate  system  being 
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evaluated.  Since  the  elementary  event  for  the  ith  criterion  was 
defined  as  (7i :Xj<X£) ,  the  intersection  or  joint  occurrence  must 
represent  the  event  that  both  (7£:Xi<x^)  and  (7j:Xj<Xj)  occur. 
Consequently,  the  probability  of  the  intersection  of  two  X^  must  be 
verbally  described  as  the  probability  that  the  raw  performance  of  the 
candidate  being  evaluated  exceeds  that  of  anv  other  candidate  system 
on  both  Cor  all)  of  the  intersecting  criteria.  Alternately,  it  may  be 
described  as  the  portion  of  the  set  of  candidate  systems  which  is 
dominated  on  both  (all)  of  the  intersecting  criteria  (Zeleney,  1977: 
150)  . 


Graphically,  we  may  add  a  dimension  for  each  intersecting 
criterion  (Figure  28).  Any  candidate  systems  falling  within  the 
doubly  shaded  region  would  be  described  as  dominated  on  both  (or  all, 
in  the  case  of  more  dimensions)  of  the  intersecting  criteria.  The 
corresponding  portion  of  the  total  set  of  candidate  systems  would  be 
the  correct  numeric  value  of  the  intersection  term  of  CF. 

Since  we  have  demonstrated  that  the  so-called  interaction  terms 
of  CF  are  really  intersection  terms,  the  question  arises,  "In  the 
design  context,  what  is  interaction  and  how  does  it  relate  to  CF?" 
Interaction  is  typically  discussed  within  the  context  of  regression  or 
linear  models  and  is  defined  (Kleinbaum,  Kupper,  and  Muller,  1988)  as 
"the  condition  where  the  relationship  of  interest  is  different  at 


Figure  28.  Dual  Criteria  Dominance 
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different  levels  (i.e.,  values)  of  the  extraneous  (control) 
variables".  Conceptually  in  the  design  context,  it  is  precisely  what 
we  have  perceived  it  to  be  all  along:  a  relationship  which  causes  the 
value  of  one  criterion  to  be  "influenced  by"  (or  "associated  with") 
the  value  of  some  other  criterion.  As  visualized  in  Figure  29,  then, 
interaction  is  any  relationship  which  would  cause  the  candidate 
systems  to  be  distributed  throughout  the  space  in  a  different 
manner,  depending  on  the  value  of  X2.  If  the  candidate  systems  were 
uniformly  distributed  throughout  the  X^,X2  space,  then  X^  and  X2  would 
be  described  as  independent  and  computation  of  X^  would  become  a 
simple  matter  of  multiplying  the  intersecting  marginal  values  (e.g. 
Xi*X2) .  However,  the  presence  of  interaction  precludes  such 
simplicity. 

For  a  moment,  let  us  pause  to  consider  the  implications  of  the 
depiction  in  Figure  28.  Such  a  figure  could  be  constructed  for  the 
evaluation  of  each  candidate  system.  Each  candidate  would  have  singly 
shaded  areas  corresponding  to  the  marginal  performance  of  that 
candidate  on  each  criterion.  These  may  be  referred  to  as  areas  of 
partial  dominance  since  they  would  contain  candidates  which  are 
dominated  on  at  least  one  criterion.  The  intersection  of  these  shaded 
areas  (the  doubly  shaded  area)  will  cover  the  portion  of  the  set  of 
candidate  systems  completely  dominated  on  the  intersecting  criteria. 
Conversely,  any  candidate  systems  falling  in  the  unshaded  area  would 
completely  dominate  the  candidate  being  evaluated.  (At  this  point,  we 
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place  no  restrictions  on  the  distribution  of  the  set  of  candidate 
systems.  Candidates  may  lie  anywhere  on  the  graph.) 

The  crux  of  the  decision  issue  then  becomes  the  ability  to 
compare  candidates  characterized  by  different  graphs  in  such  a  way 
that  the  "best”  is  chosen.  Clearly,  the  criterion  function  attempts 
to  achieve  this  by  computing  the  probability  associated  with  the 
covered  area.  Since  the  probability  of  the  doubly-shaded  area  would 
be  added  twice  by  simply  summing  the  marginal  probabilities,  it  is 
subtracted  once  so  that  it  is  only  counted  a  single  time.  This  same 
approach  is  then  easily  extended  to  additional  dimensions  for  the  case 
of  multiple  (n>2)  criteria,  although  visualization  rapidly  becomes 
difficult.  But  what  is  the  logical  rationale  of  the  CF  in  summing 
areas  of  partial  and  total  dominance?  We  shall  defer  the  answer  to 
that  question  until  after  additional  developments. 


Note  that  if  F  is  a  subset  of  E,  then  P[F]:SP[E]  (Parzen,  1960: 
20) .  Parzen  has  termed  this  relation  "the  inequality  for  the 
probability  of  a  subevent."  Since  an  intersection  event  is  a  subevent 
of  its  associated  marginal  events,  the  probability  of  any  intersection 
must  always  be  less  than  or  equal  to  the  probability  of  each 
associated  marginal  event.  This  is  logical  since  an  intersection 
requires  the  simultaneous  occurrence  of  all  relevant  marginal  events. 
For  the  simple  case  of  two  events,  we  state  that 
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P(A  B)  <  min{ P(A) , P(B) }  Eq.  4.30 

Sage  (1981)  refers  to  this  simple  relation  as  the  conjunction  rule. 
It  is  precisely  this  relation  which  provides  our  first  tangible  clue 
as  to  a  source  of  inconsistencies  in  past  implementations  of  the 
criterion  function  approach.  Specifically,  Peschke's  (1986:  133) 
demonstration  of  Model  V  encountered  Xjj  which  exceeded  one  of  their 
marginal  or  lower  order  interaction  criterion  values  (X^,  Xj  ,  etc.) 
Although  it  is  possible  that  these  results  were  isolated  anomalies, 
the  following  paragraphs  substantiate  the  contrary. 

The  approach  followed  to  compute  interaction  term  values  was 
described  earlier  in  this  chapter.  Essentially,  the  approach  consists 
of 


1)  characterizing  the  interaction  between  criteria  as  some 
function  g(x^,X2), 

2)  converting  this  function  to  a  c.d.f.  of  the  form  F(x^,X2|x^), 
and  then 

3)  using  Bayes's  rule  to  produce  F(x^,X2)  by  multiplying 
F(x1,x2|x1)  by  F(x1). 

Ostrofsky  (1968)  first  suggested  this  innovative  approach  as  a 
means  to  the  measurement  of  criteria  interactions  given  the 
limitations  of  existing  state  of  the  art  in  statistical  methods.  He 
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perceived  that  the  achievement  of  a  two-dimensional  projection  of  the 
multivariate  probability  density  function  might  facilitate  curve¬ 
fitting  via  the  simple,  yet  effective  Kolmogorov- Smirnov  test. 
Unfortunately,  the  following  evidence  indicates  inconsistencies  in 
this  method  of  computing  interactions. 

Note  that  both  of  the  functions  FCx^)  and  F(x^,X2|x^)  are  non¬ 
decreasing  as  x^  increases,  since  this  is  a  necessary  characteristic 
of  any  cumulative  distribution  (Parzen,  1960:  173).  Since  F(x^,X2>  is 
computed  as  the  product  of  these  quantities,  this  means,  in  turn,  that 
F(x^,X2>  is  non- decreasing  as  x^  increases.  This  result  is 
undesirable  since  as  F(x^)  increases,  F(x^,X2>  may  easily  exceed  F(x2> 
for  a  given  candidate  system  in  violation  of  the  conjunction  rule. 
Such  a  situation  is  depicted  in  Figure  30.  Note  that,  in  the  example, 
as  F(x^)  exceeds  .5,  the  associated  values  of  F(x2>  would  begin  to 
decrease.  Correspondingly,  the  area  of  intersection  begins  to 
decrease.  However,  using  the  current  method  of  computing  F(x^,X2),  we 
can  see  that  as  F(x^)  approaches  1.0,  F(x^,X2|x^)  also  approaches  1.0 
and,  therefore,  so  does  their  product,  F(x^)*F(x^,X2|x^) ,  which  is 
currently  perceived  to  be  F(x^,X2>.  Consequently,  we  see  that  in  this 
case,  the  current  method  causes  F(xj,X2)  to  continuously  approach  1.0 
even  though  F(x2>  is  approaching  0.  Clearly,  at  some  point,  F(x^,X2> 
will  exceed  F(x2) .  Therefore,  we  see  that  any  function  representing 
the  intersection  which  is  non-decreasing  in  x^  is  inconsistent  with 
the  conjunction  rule. 
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Furthermore,  note  that  a  c.d.f.  is  a  cumulative  representation  of 
a  density  function.  Note  further  that  a  univariate  density  function 
requires  a  two-dimensional  visual  representation  (one  dimension  for 
observed  values,  one  for  frequencies).  Similarly,  a  bivariate  density 
function  requires  a  three-dimensional  visual  representation  (one 
dimension  for  each  variable  or  criterion  plus  one  for  the 
frequencies).  Consequently,  the  curve  g(x^,X2)  in  the  two-dimensional 
Figure  30  does  not  contain  all  the  information  of  the  density  function 
f(x^,X2).  Instead,  g(x^,X2)  is  a  two-dimensional  projection  of  the 
multivariate  probability  density  function  f(x^,X2).  The  two 
dimensions  represented  are  x^  and  X2 .  The  third  (and  missing) 
dimension  is  that  of  frequency .  In  order  to  represent  a  three 
dimensional  function,  what  is  required  is  a  two-dimensional  projection 
given  the  third  dimension  (i.e.  at  a  given  frequency),  e.g.  AB|C.  By 
contrast,  the  current  method  uses  the  same  dimension  (x^)  twice,  e.g. 
AB|A.  Therefore,  the  dimension  of  relative  frequency  is  lost.  This 
accounts  for  the  inappropriate  nondecreasing  behavior  described  above. 

Measurement  of  Criteria  Intersections 

If  the  existing  method  of  computing  intersections  and/or 
interactions  is  inadequate,  what  then  is  the  correct  method?  First, 
it  is  important  to  reiterate  that,  as  demonstrated  previously,  the 
entity  required  by  the  criterion  function  and  represented  by  the  term 
Xjj  is,  in  fact,  the  intersection  of  and  X j  .  The  presence  of 
interaction  simply  complicates  the  computation  of  intersection  values. 
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As  stated  previously,  the  probability  of  an  intersection  of 
events  is  frequently  called  a  joint  probability  distribution  (Feller, 
1966:  67).  Since  we  seek  to  employ  the  cumulative  distribution 
function  as  a  measure  of  criteria  performance,  our  goal  is  then  to 
obtain  the  joint  cumulative  probability  distribution  function  of  the 
intersecting  criteria. 

Feller  (1966)  introduces  multivariate  distributions  beginning 
with  consideration  of  the  case  of  a  two-dimensional  Cartesian  plane. 
He  states  that  for  any  region  0,  the  density  f  attributes  the 
probability 
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J  ^  f(x1,x2)dx1dx2 
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Eq.  4.31 


For  a  specific  area  bounded  by  the  values  a^,  b^  in  the  x^  dimension 
and  a2,  b2  in  the  x2  dimension  we  have 


P ( a^<x^<b^ , a2<x2£b2 ) 


blb2 

55 

ala2 


f (x^ ,x2)dx^dx2 


Eq.  4.32 


for  all  combinations  where  a^<b^.  By  letting  a^-a2--«  we  obtain  the 
desired  distribution  function  F  of  f 


F(xlfx2)  -  P(X1<x1,X2<x2) 


Eq.  4.33 
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Note  that  this  relationship  was  clearly  recognized  and  described  by 
Ostrofsky  (1968,  1977)  (See  Appendix  A  of  this  dissertation,  Equations 
A. 43  and  A. 56.)  Presumably,  however,  recognition  of  its  significance 
became  overshadowed  in  the  effort  to  achieve  the  best  possible  means 
of  representing  a  continuous  multivariate  density  function  to 
facilitate  the  measurement  of  interactions.  However,  as  the  preceding 
discussion  reveals,  the  method  based  on  two-dimensional  projections  is 
capable  of  producing  results  which  are  inconsistent  with  the 
conjunction  rule.  We  are  therefore  forced  to  return  to  the  best 
conventional  means  available  for  representing  a  density  function  in 
order  to  permit  the  use  of  Equation  4.35  for  the  computation  of 
intersection  terms. 


When  criteria  are  independent,  the  computation  of  intersection 


values  is  greatly  simplified.  In  this  case,  we  have  (Parzen,  1960: 
295;  Ross,  1988) 


F(x^,X2)  -  P{Xi<xi,  X£^X2)  Eq.  4.36 

-  P(X1<x1)*P{X2^X2^  Eq.  4-37 

-  Fxl(xl^*Ex2^x2^  ^or  xl>x2  Eq-  4-38 


More  generally,  for  the  case  of  n  independent  criteria 


n 

F(x1,X2 . Xp)  -  it  P{Xj^X£)  Eq.  4.39 

i-1 


n 

-  it  F^(X£)  Eq.  4.40 

i-1 


Visualized  graphically  in  three  dimensions,  the  intersection  of 
two  independent  criteria  might  appear  as  in  Figure  31.  Since  the 
criteria  are  independent,  the  density  function  is  the  same  in  the  X^ 
space,  regardless  of  changes  in  X2  (and  vice  versa).  Recall  now  that 
when  criteria  interact,  the  relationship  of  interest  (the  density 
function,  f(x^,X2))  is  different  at  different  values  of  x^  and  X2- 
Graphically,  interaction  might  produce  a  density  function  of  the  sort 
shown  in  Figure  32.  No  longer  can  the  intersection  value  F(x^,X2)  be 
computed  simply  as  F^(x^)*F2(x2) .  Instead,  it  will  be  necessary  to 
formulate  a  continuous  density  function,  f(x^,X2)  and  compute  Xjj 
using  Equation  4.31  (or,  more  generally,  Equation  4.35).  Therefore, 
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Note  that  in  some  cases,  it  may  not  be  possible  to  fit  a 
continuous  density  function  even  though  the  presence  of  interaction  is 
too  great  to  ignore.  In  such  cases,  we  may  choose  to  treat  the 
distribution  of  candidate  systems  as  discrete  and  accurately 
represented  by  the  distribution  of  identified  candidate  systems. 
Given  these  assumptions,  we  could  then  compute  intersections  by  simply 
counting  the  number  of  candidate  systems  dominated  on  all  of  the 
intersecting  criteria  and  dividing  by  the  total  number  of  candidate 
systems  to  produce  the  desired  proportion. 

An  important  implication  of  this  "rediscovery"  (although 
tangential  to  this  specific  research) ,  is  the  implicit  assertion  that 
even  Models  I -IV  of  the  criterion  function  should  include  intersection 
terms .  Models  V-VIII  would  still  be  distinguished  by  the  presence  of 
interaction.  However,  the  difference  between  the  two  classes  of 
models  would  no  longer  lie  in  the  presence  or  absence  of  intersection 
terms,  but  rather  in  the  manner  in  which  those  terms  are  computed.  In 
other  words,  Models  I -IV  have  intersection  terms  while  Models  V-VIII 
have  intersection  terms  with  interaction.  Therefore,  the  criterion 
function  for  all  eight  models  would  be  computed  using  Equation  4.6 
including  intersection  terms  (i.e.  all  S-l) .  However,  for  Models  I-IV 
(independent  criteria),  Equation  4.40  would  be  used  to  compute  the  X^j 
intersection  terms,  while  Models  V-VIII  (interacting  criteria)  would 
require  intersection  computation  via  Equation  4.35.  Since  the  latter 
case  may  require  the  estimation  of  a  continuous  multivariate  density 
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function,  regression  techniques  and/or  use  of  a  Chi-Square  or  more 
robust  tests  may  be  required. 

Further  support  of  the  need  for  (independent)  intersection  terms 
may  be  found  in  Parzen's  (1960:  88)  observation  that  "two  mutually 
exclusive  events,  A  and  B,  are  independent  if  and  only  if  P[A]P[B]-0, 
which  is  so  if  and  only  if  either  A  or  B  has  probability  zero."  In 
out  context,  this  means  that  for  two  design  criteria  to  be  both 
mutually  exclusive  and  independent,  the  probability  associated  with  at 
least  one  of  the  criteria  must  be  0  when  the  other  exists.  Under  our 
current  assumptions,  this  should  not  occur.  Therefore,  either  the 
criteria  are  not  mutually  exclusive  o£  they  are  not  independent. 
Since  we  accept  the  fact  of  their  independence  (this  is  the  very  basis 
for  the  selection  of  Models  I-IV  as  opposed  to  Models  V-VIII),  we 
conclude  that  they  must  not  be  mutually  exclusive  (i.e.  they 
intersect).  Consequently,  the  computation  of  their  union  requires  the 
subtraction  of  their  intersection. 

In  reexamining  the  revised  computation  of  intersection  terms ,  it 
should  be  noted  that  using  Equations  4.35  and  4.40,  X^j  will  always  be 
less  than  or  equal  to  the  marginal  criterion  values,  and  Xj .  This 
is  easily  seen  graphically  in  Figures  31  and  32.  Since  X^j  is 
effectively  a  subset  of  X^ ,  it  can  never  be  greater  than  X^ . 
Therefore,  we  have  achieved  compliance  with  the  conjunction  rule 
(Equation  4.30).  What,  however,  has  been  the  impact  on  the  value  of 
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CF  itself  and,  more  importantly,  have  we  completely  solved  the 
observed  discrepancies  in  CF  values? 

To  examine  the  effect  of  intersection  terms  on  CF,  consider  the 
trivial  case  of  two  independent  criteria  of  equal  relative  importance. 

Now  assume  the  existence  of  two  candidate  systems  (A,B)  whose 
criterion  values  are  X^  -  .1,  X2  -  .9  for  A  and  X^  -  .5,  X2  -  .5  for 
B.  Initial  inspection  might  infer  that  since  the  two  criteria  are 
equally  important,  these  two  candidate  systems  are  equally  preferable. 

To  see  this,  we  define  isopreference  lines  to  be  lines  which  connect 
points  of  equal  preference  (a  la  Keeney  and  Raiffa,  1976).  Then  for 
two  equally  important  independent  criteria,  we  might  expect 
isopreference  lines  to  be  a  family  of  straight  lines  of  the  form  X^  + 

X2  -  C  where  0<C<2.0.  These  lines  would  appear  as  in  Figure  33  and, 
as  suggested,  candidates  A  and  B  would  seem  to  fall  on  the  same  line, 
indicating  equal  preference. 

Given  that  the  marginal  relative  weights  are  equal,  the  sum  of 
the  marginal  6^  will  be  equal  for  the  two  candidate  systems  (due  to 
the  commutative  property  of  multiplication)  since  the  sum  of  the 
marginal  X^  equals  1.0  in  both  cases.  Now  recall  that  since  X^  and  X2 
are  independent,  the  intersection  terms,  X]_2»  °f  A  and  B  are  computed 
as  .1*.9  -  .09  and  .5*. 5  -  .25,  respectively.  Since  the  intersection 
term  is  subtracted  in  CF  (Equation  4.6)  and  since  the  relative 
weights,  a^  and  a2 ,  will  be  equal,  we  can  see  that  for  any  positive 
value  of  a^2>  candidate  A  will  be  preferred  to  candidate  B  because  B 
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Figure  33.  Isopreference  Lines 
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has  a  larger  intersection  term.  In  terns  of  isopreference  lines, 
subtracting  interaction  terms  clearly  makes  the  lines  convex  (e.g. 
Figure  34)  where  the  degree  of  curvature  would  be  determined  by  the 
magnitude  of  a^,  thus  redefining  the  isopreference  lines.  Since  an 
inverse  relationship  between  the  size  of  the  intersection  term  and  the 
value  of  CF  may  seem  counterintuitive,  we  seek  to  understand  the 
underlying  (theoretical)  rationale. 

The  reason  for  this  behavior  can  be  seen  by  examining  the  areas 
of  dominance  associated  with  the  two  candidate  systems.  Candidate  A 
(Figure  35)  has  larger  areas  of  partial  dominance  and  a  smaller  area 
of  complete  dominance  (intersection).  By  contrast,  candidate  B 
(Figure  36)  has  smaller  areas  of  partial  dominance  and  a  larger  area 
of  complete  of  complete  dominance.  Intuitively,  we  might  suspect  that 
B's  larger  area  of  complete  dominance  would  make  it  preferred. 

However,  recall  that  CF  selects  the  candidate  with  the  largest 
total  area  of  the  graph  covered  (via  either  partial  o£  complete 
dominance) .  Recognize  that  as  the  covered  portion  of  the  graph  gets 
larger,  the  uncovered  portion  gets  smaller.  Since  the  covered 
portior  represent  those  candidates  which  are  dominated  by  the  one 
being  evaluated  on  at  least  one  criterion,  the  uncovered  area 
represents  undominated  candidates,  or,  more  to  the  point,  candidates 
which  would  dominate  the  one  being  evaluated.  Therefore,  we  can  see 
that  the  evaluation  accomplished  by  CF  is  best  understood  by  examining 


the  behavior  of  the  uncovered  rather  than  the  covered  area. 
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Figure  34.  Isopreference  Curves  Caused  by 
Subtraction  of  Intersections 
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Figure  35.  Areas  of  Dominance  f  .*r  Candidate  A 
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Figure  36.  Areas  of  Dominance  for  Candidate  B 
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Specifically,  it  can  be  seen  that  CF  favors  the  candidate  system  which 
minimizes  the  probability  that  it  might  be  completely  dominated  bv 
some  other  system  (1  -  CF) . 

Observe  how  this  understanding  parallels  the  earlier,  more 
general  discussion  of  the  rationale  for  the  form  of  the  criterion 
function  in  terms  of  8 £.  Specifically,  we  recognize  that  the  behavior 
of  the  criterion  function  is  consistent  with  the  concept  of  attempting 
to  minimize  the  probability  of  a  bad  choice.  In  other  words,  we  seek 
the  candidate  system  which  is  "farthest"  from  the  worst  possible 
choice.  Therefore,  candidates  which  are  equally  bad  are 
indistinguishable.  Figure  34  supports  this  interpretation,  showing 
that  (in  the  case  of  equal  relative  weights)  points  which  are 
equidistant  from  the  origin  (the  worst  possible  choice)  share  the  same 
isopreference  curve.  (By  contrast,  decision  making  approaches  which 
seek  to  get  as  close  as  possible  to  the  theoretical  ideal  would 
reflect  isopreference  curves  composed  of  points  equidistant  from  the 
upper  right-hand  corner  instead  of  the  origin.) 

With  this  understanding,  the  performance  of  CF  with  respect  to 
our  example  candidates  A  and  B  becomes  quite  logical.  Note  that 
although  B's  area  of  total  dominance  is  larger  than  A's,  the  increase 
is  achieved  at  the  expense  of  the  uncovered  area.  In  other  words, 
candidate  B  has  a  greater  probability  that  some  other  candidate  exists 
which  dominates  B  on  all  criteria.  Simplistically ,  there  is  a  greater 
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chance  of  finding  a  system  which  is  indisputably  superior  to  B  than  of 
finding  one  superior  to  A. 

Although  we  can  now  logically  explain  the  behavior  of  CF,  can  we 
be  sure  it  will  remain  in  the  [0,1]  range?  Examination  of  Equation 
4.6  (the  criterion  function  with  relative  weights)  reveals  that  we  can 
make  no  such  assertion  without  additional  knowledge  of  the 
relationships  among  the  marginal  and  intersection  relative  weights 
(a^,  a^j  ,  etc.).  Although  we  now  know  that  Xjj  will  always  be  less 
than  or  equal  to  the  associated  X^,  it  is  still  possible  that  a^j 
could  be  so  large  compared  to  the  a^  that  the  overall  value  of  CF 
becomes  negative.  Therefore,  we  examine  the  inclusion  of  relative 
weights  in  the  next  section. 


Inclusion  of  Relative  Weights 

Intersection  Relative  Weights 

Before  proceeding,  we  acknowledge  that  the  notation  used  so  far 
for  the  representation  of  relative  weights  has  been  somewhat  loosely 
specified.  Prior  to  this  point,  we  have  attempted  to  remain 
consistent  with  the  previous  work  being  discussed.  However,  from  this 
point  on,  we  implement  the  following  standard  notation  and  recommend 
its  continued  use  in  future  research: 


Let 
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a i  -  the  relative  weight  of  the  ith  criterion  as 
originally  specified  by  the  decision  maker 

a^'  -  the  ith  relative  weight  following  "vertical" 
normalization 

aj/ '  -  the  ith  relative  weight  following  both 

"vertical"  and  "horizontal"  normalization 

A£  -  the  ith  relative  weight  as  finally  included 
in  the  computation  of  CF 

A  second  subscript  may  be  used  to  denote  the  value  for  a  specific 
candidate  system  and  a  superscript  may  be  used  to  indicate  the 
relative  weight  within  a  discrete  interval  of  the  range .  Note  that 
under  current  practice,  A^-aj/ ' .  However,  the  following  section 
proposes  changes  in  this  regard. 

Ostrofsky  (1977d,  1987,  1988)  has  defined  relative  weights  {a^} 
as  subjective  measures  of  the  relative  importance  (usually  subjective) 
of  the  decision  criteria.  As  have  Fishburn  (1970)  and  Wymore  (1976), 
he  has  also  suggested  (1977d)  that  relative  weights  may  be  treated  as 
subjective  probabilities.  Consequently,  current  practice  in 

implementing  the  criterion  function  (Ostrofsky,  1977d,  1987)  requires 
that  all  relative  weights  sum  to  1.0.  Since  intersection  (heretofore 
"interaction")  terms  must  be  considered  events  in  their  own  right 
(Loeve,  1963),  it  has  been  required  that  relative  weights  be  assigned 
to  them  as  well  (e.g.  Feschke,  1986)  under  the  pervasive  restriction 
that  the  sum  of  all  relative  weights  equals  1.0.  For  example,  in  a 
three-criterion  case,  the  relative  weights  in  Table  10  would  be 
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TABLE  10. 

EXAMPLE  RELATIVE  WEIGHTS 


Criteria  Relative  l 

X1  .1 

X2  .05 

X3  .2 

*12  -1 

X13  .2 

X23  .2 

Xl23  -15 


1.00 


considered  an  acceptable  combination.  These  weights  could  be 
visualized  in  the  form  of  a  Venn  diagram  (Figure  37). 
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We  have  previously  shown  that,  as  probabilities,  the  8 ^  and 
are  both  required  to  satisfy  the  conjunction  rule.  Since  the  relative 
weights  are  probabilities  in  their  own  right,  we  know  that  the 
following  conjunction  relation  must  also  be  satisfied: 

PCajj)  £  min{P(a£> ,  P(aj))  Eq.  4.41 

Yet,  as  currently  implemented,  there  is  no  assurance  that  this 
relationship  will  be  satisfied  (e.g.  Peschke,  1986:  90).  In  order  to 
avoid  its  violation,  there  might  be  a  temptation  to  simply  add  a 
procedural  constraint  that  Equation  4.41  must  be  satisfied  in  the 
initial  definition  of  relative  weights.  However,  as  would  have  been 
the  case  with  the  functional  interaction  terms,  such  a  stopgap  effort 
would  serve  only  to  obscure  an  underlying  source  of  inconsistency 
which  requires  revision. 

Reexamination  of  the  Venn  diagram  (Figure  37)  reveals  that, 
contrary  to  current  practice,  the  marginal  relative  weights  used  in 
the  computation  of  CF  should  not  be  as  defined  in  Table  10.  Rather, 
the  complete  marginal  relative  weight  consists  of  the  sum  of  the 
events  which  comprise  it.  In  other  words,  the  relative  weight  of  a 
ffl&rginel  criterion _ (when  placed  into  the  criterion  function)  must 


part .  Similarly,  the  relative  weight  of  an  intersection  term  must 
include  the  relative  weights  of  all  higher  order  intersection  terms 
included  within  it.  As  a  generic  relationship,  we  state  that 
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Ai 


a"  l  +  S  a"ii 

j-1 

i/j 


n  n 


n  n 


+  Z  2  a"ijk  +•••+  S...S  a"i  x  Eq.  4.42 
j-1  k-1  j-1  1-1 

i/j  1*1 

i/k 

j*k 

1-1/1 


Note  that  this  realization  infers  that  the  a^' '  as  currently  elicited 
and  used  do  not  include  the  a^j ' ' .  (This  fact  is  seen  in  Figure  37 . ) 
Therefore,  the  aij ' '  are  not  currently  the  Boolean  intersections  of 
the  a^' '  and  a j ' '  .  However,  with  the  additional  operation  of  Eq. 
4.42,  the  Ajj  do,  in  fact,  become  the  intersections  of  the  A^  and  A  j  , 
satisfying  the  conjunction  rule  (Eq.  4.41). 


This  relationship,  combined  with  the  previous  discussion 
concerning  the  X^j ,  leads  to  substantially  improved  insight  concerning 
the  meaning  of  the  ajj  as  currently  defined.  Since  the  Xjj  represent 
the  measure  of  performance  on  criteria  i  and  j  simultaneously,  th<'  a^j 
can  be  seen  to  represent  the  Importance  of  a  balance  in  performance 
among  the  intersecting  criteria  (possibly  achieved  at  the  expense  of 
performance  on  marginal  criteria  values).  Since,  for  any  particular 
application,  no  specific  relationship  may  be  assumed  a  priori 
concerning  the  importance  of  marginal  criteria  vs.  criteria  balance, 
it  is  not  appropriate  to  mandate  any  particular  relationship  between 
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the  and  the  a^j  ,  other  than  that  they  sum  to  1.0  (as  achieved  via 
normalization) .  Therefore,  the  a^j  may  be  either  greater  than  or  less 
than  the  a^.  However,  Equation  4.42  is  required  to  reflect  the 
hierarchical  relationship  among  intersection  terms  which  becomes 
necessary  when  relative  weights  are  collectively  placed  in  the  context 
of  the  criterion  function.  In  other  words,  Eq.  4.42  is  the  means  by 
which  the  independently  identified  a^  and  ajj  are  made  consistent  with 
the  laws  of  probability  intersections  and,  therefore,  with  the  and 
Xij. 


Therefore,  for  the  purposes  of  computing  CF  in  the  example 
problem,  we  would  revise  the  relative  weights  as  shown  in  Table  11. 
Note  that  this  approach  for  handling  a^  is  consistent  with  the 
definition  of  the  X's  in  which  X^  and  Xj  each  include  X^j . 

As  partial  support  for  the  validity  of  this  proposed  change, 
consider  candidate  system  performance  at  both  ends  of  the  spectrum. 
It  is  expected  that  for  the  worst  theoretically  possible  candidate 
(all  Xj_-0) ,  CF  should  be  0  and  for  the  theoretic  ideal  (all  Xj-1) ,  CF 
should  equal  1.  For  the  worst  candidate,  both  the  current  and 
proposed  methods  would,  as  desired,  yield  a  value  of  0.  However,  for 
the  best  possible  system,  the  current  method  may  well  produce  some 
value  less  than  1,  since  all  A ^  sum  to  1.0  yet  some  Ajj  are  subtracted 
rather  than  added.  If  the  sum  of  the  subtracted  relative  weights  is 
greater  than  that  of  those  added,  CF  may  even  become  negative.  By 
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TABLE  11. 

REVISED  RELATIVE  WEIGHTS 


Criteria 

Xl 

x2 

X3 

Xi2 

Xl3 

X23 

xl23 


Relative  Weights 
.1+.1+.2+.15-.55 
.05+.1+.2+.15-.5 
.2+.2+.2+.15-.75 
.1+.15-.25 
.2+.15-.35 
.2+.15-.35 


.15 


contrast,  the  proposed  method  would  produce  CF-1  as  desired 
regardless  of  the  values  of  the  relative  weights. 
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The  presence  of  the  a^j  and  their  distinction  from  the  a^ 
constitute  one  of  the  more  powerful  and  unique  characteristics  of  the 
criterion  function  approach  to  decision  making.  While  most  other 
approaches  are  based  on  some  specific  combination  of  marginal  criteria 
performance,  the  criterion  function  permits  explicit  tradeoffs  among 
not  only  marginal  criteria,  but  also  all  possible  criteria 
combinations . 


Merging  Criteria  Measures  and  Relative  Weights 

Finally,  it  remains  to  consider  the  merger  of  the  "objective" 
{Xj^  and  "subjective"  {AjJ  terms  in  the  form  of  the  CF.  Ostrofsky 
(1968,  1977d)  logically  considers  this  merger  in  terms  of  the 
intersection  of  the  two  probability  measures  associated  with  each 
event.  In  the  simplest  case  of  constant  relative  weights,  Ostrofsky 
(1977d)  observed  that  the  values  of  and  X^  are  independent: 
knowledge  of  the  value  of  one  has  no  effect  on  the  value  of  the  other. 
Therefore,  the  intersection  is  simply  formed  as  AjX^  and  the  criterion 
function  emerges  as  the  union  of  these  intersections. 

Clearly,  in  the  case  of  varying  relative  weights  as  we  have 
defined  them,  the  assumption  of  independence  between  the  criteria 
values  and  their  relative  weights  is  no  longer  valid.  Now,  the  value 
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of  is  intentionally  influenced  by  the  value  of  X^.  Fortunately, 
the  method  suggested  for  defining  relative  weights  produces  - 
f(X^) .  Therefore,  the  resulting  weight  is  actually  A^|X^  and  not 
simply  A^.  Since 


P(AAB)  -  P(A)*P(B | A)  Eq.  4.43 

we  can  still  multiply  the  two  values  directly  as 

PCajflXi)  -  P(ai|xi)*P(xi)  Eq.  4.44 

where  P(a^|x£)  is  simply  the  normalized  value  of  the  relative  weight 
and  P(x^)  is  X^.  So  although  the  underlying  rationale  is  different 
from  the  case  of  constant  relative  weights,  the  mechanics  of  merging 
the  {A^}  and  {X^}  remain  essentially  the  same.  For  simplicity's  sake, 
we  choose  to  ignore  the  distinction  henceforth,  using  the  symbol  A^  in 
lieu  of  the  more  accurate  (AjJX^). 

Normalization  of  Continuously  Variable  Relative  Weights 

The  vertical  and  horizontal  relative  weight  normalization  scheme 
of  Ostrofsky  (1987)  is  a  truly  subtle  and  remarkable  mechanism, 
meriting  further  study  beyond  the  scope  of  this  research.  Note  that 
without  explicitly  defining  the  complete  preference  structure  of  the 
decision  maker,  normalizing  the  (a^)  first  vertically  and  then 


horizontally  provides  a  means  of  implicitly  responding  to  the  presence 
of  preferential  interaction  across  criteria. 

For  the  case  of  relative  weights  which  remain  constant  within 
intervals  but  vary  between  intervals  (Models  II  and  VI),  vertical 
normalization  was  achieved  prior  to  evaluation  of  any  specific 
candidate  systems.  Specifically,  the  relative  weights  in  each 
interval  were  normalized  a  priori.  These  normalized  weights  were  then 
used  to  evaluate  CF  for  each  candidate.  The  extension  to  the  case  of 
continuously  varying  relative  weights  (Models  III  and  VII)  then 
becomes  a  simple  matter  of  revising  the  sequence,  but  not  the  content, 
of  the  vertical  normalization  scheme.  The  revised  algorithm  for 
determining  relative  weights  is  as  follows: 

A)  Determine  the  function,  a^  -  f^(X^)  for  each  i. 

B)  For  each  candidate  system: 

1)  For  each  i: 

a)  Compute  a^  -  fi(X^) 

b)  Compute  aj  -  f j (X^)  for  each  j  at  that  value 
of  i 

c)  Compute  the  vertically  normalized  value  of  a^ 
as 

a^'  -  a^/Za^  Eq.  4.45 

J 

2)  Repeat  step  1  for  each  criterion. 

3)  Horizontally  normalize  the  a^  for  each  candidate 
system  to  produce  {aj/'}. 
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4)  Revise  relative  weights  to  include  related  higher 
order  intersection  relative  weights  using  Equation 
4.42,  producing  {A^}. 

C)  If  more  candidate  systems  remain,  return  to  step  B. 

In  sum,  the  revision  simply  defers  vertical  normalization  until  a 
specific  value  within  the  X^  range  is  identified  for  a  given 
combination  of  candidate  system  and  criterion.  The  obvious  rationale 
for  this  revision  in  sequence  is  that  it  would  be  impossible  to 
accomplish  vertical  normalization  a  priori  with  continuously  varying 
relative  weights  since  there  are,  effectively,  an  infinite  number  of 
intervals  in  which  a^  would  have  to  be  computed.  Consequently,  the 
determination  of  which  intervals  (points)  should  be  normalized  is 
deferred  until  it  is  known  which  points  are  needed.  An  example  of  the 
revised  method  is  shown  in  Figure  38. 

Empirical  vs.  Continuous  Distribution  Functions 

Throughout  this  chapter  we  have  generally  assumed  that  the 
cumulative  distribution  function  used  to  produce  the  X^  from  the  x^ 
will  take  the  form  of  a  continuous  (probably  nonlinear)  function. 
This  section  discusses  those  cases  in  which  we  are  unable  or  unwilling 
to  use  such  a  function. 

The  identification  of  a  continuous  c.d.f.  would  require  the 
definition  of  a  continuous  function  which  "reasonably"  approximates 
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Figure  38.  Normalizing  Continuously  Variable  Relative  Weights 
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the  empirical  data  points  observed.  The  enforcement  of  the 
"reasonableness"  characteristic  could  be  achieved  through  the  use  of 
an  established  statistical  test,  e.g.  Kolmogorov- Smirnov,  Chi-Square, 


etc.  However,  it  is  also  possible  (particularly  in  the  case  of 
complex  interaction)  that  no  function  can  be  identified  which  is  able 
to  "pass"  such  objective  tests.  When  this  condition  occurs,  the 
decision  maker  faces  at  least  three  options. 

One  option  is  to  assume  independence  among  criteria.  This  option 
may  be  preferred  in  cases  where  visual  partial  plots  of  the  data 
points  present  no  consistent  appearance  of  meaningful  interaction. 
(This  was  the  case  with  respect  to  functional  interaction  among  the 
data  used  by  Folkeson  (1982).)  Although  it  may  not  be  technically 
accurate  to  assume  that  the  criteria  are  truly  independent  in  such  a 
case,  the  premise  is  that  if  we  are  unable  to  clearly  define  or  make 
sense  of  the  nature  of  the  interaction,  we  may  be  safer  in  ignoring 
it.  An  analogy  might  be  that  simple,  naive  forecasting  models 
frequently  perform  better  than  more  sophisticated  models  which  assume 
the  wrong  underlying  model  (Makridakis,  et  al,  1982). 

Another  option  is  to  simply  fit  the  best  continuous  function 
possible  and  use  it.  The  essence  of  this  approach  is  simply  doing  the 
best  one  can  with  real  world  data.  Curve -fitting  techniques  (i.e. 
regression)  could  be  used  with  the  intent  of  minimizing  or  maximizing 
some  model  selection  criteria,  e.g.  ,  S^y|x,  F  statistic,  MSE, 
Mallow's  Cp,  PRESS,  etc.  (Kleinbaum,  Kupper,  and  Muller,  1988).  By 
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fitting  a  variety  of  models  for  different  combinations  of  criteria  and 
bv  considering  the  intuitive  nature  of  the  potential  interactions,  the 
decision  maker  may  come  to  some  conclusion  concerning  which 
interactions  merit  modeling,  even  at  the  risk  of  inaccuracy. 

A  third  alternative  is  to  use  the  empirical  distribution  rather 
than  a  continuous  approximation.  The  remaining  paragraphs  of  this 
section  address  this  alternative . 

Before  proceeding  with  a  discussion  of  the  use  of  empirical 
distributions,  we  suggest  that  a  strong  case  can  be  made  for  the  use 
of  empirical  distributions  even  when  approximation  with  a  continuous 
function  is  feasible.  First,  recognize  that  applications  of  the 
criterion  function  could  be  divided  into  two  general  cases :  those 
instances  where  the  defined  set  of  candidate  systems  is  highly 
speculative  or  known  to  be  incomplete  and  those  instances  where  the 
candidate  systems  are  well  defined  or  firm  and  we  do  not  wish  to 
consider  any  additional  alternatives.  In  the  former  case,  we 
effectively  assume  an  infinite  number  of  candidate  systems.  One  of 
the  primary  advantages  of  the  continuous  c.d.f.  in  this  case  is  that 
its  use  in  the  search  of  the  design  space  may  lead  to  the  discovery  of 
previously  unrecognized,  and  potentially  superior  alternatives. 
Therefore,  use  of  a  continuous  c.d.f.  may  be  highly  beneficial. 
However,  in  the  latter  case,  we  are  not  interested  in  the  discovery  of 
new  candidates.  Instead,  our  focus  is  solely  on  the  determination  of 
the  relative  best  from  among  those  alternatives  already  specified.  In 
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such  a  case,  our  real  problem  is  representing  the  empirical  data  as 
accurately  as  possible.  In  such  a  case,  the  use  of  a  continuous 
approximation  may  unnecessarily  sacrifice  accuracy.  Therefore,  use  of 
the  empirical  distribution  may  be  the  most  appropriate  course  of 
action.  (In  fact,  in  such  a  case  one  could  well  argue  that  no  large 
minimum  number  of  empirical  data  points  must  be  available  since  no 
statistical  inference  or  estimation  will  be  attempted.  Instead, 
probability  is  serving  only  as  an  objective  guide  to  the  evaluation 
effort. ) 

Feller  (1966:  36)  defines  the  empirical  distribution  function  Fn 
of  n  points  a^,  a2>...,  an  as  the  step  function  with  jumps  of  1/n  (on 
the  y-axis)  at  points  a^,  a2,...,  ^  (on  the  x-axis).  Given  this 
definition  and  the  fact  that  the  sample  description  space  is  now 
considered  finite,  the  computation  of  the  probability  of  an  event  is 
quite  straightforward.  For  any  event  E  on  the  sample  space  S,  we  have 
that  (Parzen,  1960:  25) 

P[E]  -  N[E]/N[S]  -  (size  of  E)/(size  of  S)  Eq.  4.46 

In  other  words,  use  of  the  empirical  distribution  function  permits  us 
to  simply  count  the  number  of  events  meeting  the  desired  description 
and  compute  that  number's  ratio  over  the  complete  set.  For  a 
particular  criterion  and  candidate  system,  then,  we  would  count  the 
number  of  candidate  systems  whose  performance  (x^)  was  equal  to  or 
worse  than  the  performance  of  the  candidate  we  are  evaluating.  We 
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would  then  divide  the  resulting  number  by  the  total  number  of 
candidate  systems  to  obtain  the  for  the  candidate  we're  evaluating. 


Similarly,  for  intersection  values  (x^j),  we  need  only  count  the 
number  of  candidates  whose  performance  is  exceeded  by  the  candidate 
we're  evaluating  on  both  (or  all)  of  the  intersecting  criteria  and, 
again,  divide  by  the  total  number  of  candidate  systems. 

Note  the  extreme  simplicity  of  using  the  empirical  distribution 
compared  to  the  continuous  function.  First,  we  need  not  concern 
ourselves  with  fitting  or  evaluating  curves  (continuous  functions) . 
Second,  we  need  no  longer  determine  the  values  of  zj  m^n,  Zj  max,  x^ 
min*  or  xi  max-  Third,  the  use  of  empirical  distributions  is  made  no 
more  difficult  by  the  presence  of  interaction.  Finally,  once  the 
computations  of  the  x^  values  are  complete,  we  are  primarily  only 
performing  counting  operations  rather  than  computations .  Therefore , 
the  evaluation  of  candidate  systems  may  well  lend  itself  to  direct 
spreadsheet  quantification  rather  than  requiring  formal  programming 
efforts,  making  it  potentially  more  appealing  to  practitioners. 

In  closing,  it  should  be  noted  that  the  intent  of  this  section 
has  not  been  to  argue  that  empirical  distribution  functions  should 
arbitrarily  be  used  as  opposed  to  continuous  functions.  On  the 
contrary,  we  reiterate  that  whichever  form  is  most  appropriate  for  the 
application  should  be  used.  However,  since  historically,  discussions 
of  the  criterion  function  approach  have  focused  on  the  continuous 
case,  this  section  was  intended  to  identify  circumstances  under  which 


use  of  the  empirical  alternative  might  be  appropriate  and  the  benefits 
which  might  be  expected  to  accrue.  As  always,  these  benefits  must  be 
weighed  against  the  corresponding  sacrifices. 
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Summary  of  Revisions 

In  the  second  half  of  this  chapter  we  have  clarified  procedural 
methods  of  past  and  proposed  implementations  of  the  criterion  function 
approach  to  design  decision  making.  In  the  process,  we  identified 
current  sources  of  inconsistency  and  identified  two  significant 
changes  in  the  methods  of  implementing  the  relevant  theory  to  resolve 
these  inconsistencies. 

First,  we  have  shown  that  the  criterion  function  requires  the 
measurement  of  criteria  intersections  and  have  distinguished  them  from 
the  concept  of  interaction.  We  have  shown  that,  when  continuous 
approximations  of  distribution  functions  are  used,  the  current  method 
of  computing  the  intersections  (Xjj)  must  be  replaced  with  either  Eq. 
4.35  or  Eq.  4.40,  depending  on  whether  or  not  interaction  is  present. 
We  subsequently  described  circumstances  under  which  the  use  of 
empirical  distribution  functions  may  be  appropriate,  in  which  case  the 
computation  of  intersections  may  be  accomplished  via  counting  of  the 
appropriate  intersection  events  and  the  use  of  Eq.  4.46. 

Second,  we  have  used  similar  reasoning  to  show  that  an  additional 
step  (Eq.  4.42)  is  required  to  include  higher-order  intersection 
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relative  weights  in  the  determination  of  A^,  satisfying  probability 
theory  in  general  and  the  conjunction  rule  in  particular.  As  part  of 
this  discussion,  we  achieved  greater  insight  to  the  meaning  and 
significance  of  intersection  relative  weights,  as  well  as  the 
flexibility  of  their  relationship  to  marginal  relative  weights.  We 
have  also  described  the  appropriate  method  for  accommodating 
continuously  variable  relative  weights. 

By  establishing  consistency  with  probability  theory,  the  proposed 
revisions  complete  the  ability  of  the  criterion  function  to  achieve 
the  stated  research  objective.  In  the  next  chapter,  we  demonstrate 
the  theoretical  contributions  provided  here  in  the  context  of  an 
actual  design  decision  problem. 


Chapter  5 


A  DESIGN  DEMONSTRATION:  SPACE  SHUTTLE 
AUXILIARY  POWER  UNITS 


Purpose  of  the  Demonstration 

The  objectives  of  this  dissertation  were  specified  at  the  ends  of 
Chapters  1  and  3 .  The  primary  focus  of  the  research  concerns  the 
theoretical  approach  of  the  criterion  function  and  the  appropriate 
methods  for  applying  that  theory.  These  issues  were  addressed  and 
resolved  in  Chapter  4.  However,  to  enhance  clarity  and  to  show  that 
the  proposals  of  Chapter  4  produce  the  desired  results,  this  chapter 
demonstrates  the  application  of  the  revised  criterion  function 
implementation  method  to  an  actual  complex  design  decision  problem: 
the  choice  of  a  design  for  auxiliary  power  units  for  NASA's  space 
shuttle . 

It  is  appropriate  to  emphasize  that  the  purpose  of  this  chapter 
is  to  demonstrate  the  proposals  presented  in  Chapter  4,  not  to 
actually  solve  NASA's  problem  since  the  data  comes  from  the  Johnson 
Space  Center.  While  the  former  could  be  accomplished  with  the 
resources  and  information  currently  available,  the  latter  would 
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necessitate  a  far  more  extensive  effort,  including  a  variety  of  spin¬ 
off  studies  and  laboratory  testing  efforts  which  could  be  made 
possible  only  with  dedicated  funding  and  manpower  allocation  by  NASA 


and  its  contractors.  Consequently,  the  data  used,  the  models 
constructed,  and  the  results  obtained  should  be  accepted  as  useful  for 
the  demonstration  purposes  intended  and  for  the  general  insight 
provided,  but  should  not  be  misconstrued  as  representing  the 
definitive  solution  to  be  pursued.  It  is  hoped,  however,  that  in 
addition  to  achieving  the  research  purposes,  the  demonstration  will 
provide  a  useful  framework  and  first  step  toward  the  ultimate  solution 
should  NASA  proceed  with  their  current  interest  in  this  problem. 

As  a  final  note,  it  is  recognized  that  an  unfortunate  aspect  of 
the  demonstration  used  is  that  it  may  unintentionally  perpetuate  the 
common  misconception  that  system  design  methods  are  applicable  only  to 
hardware  systems.  Therefore,  the  reader  is  encouraged  to  observe  the 
generic  nature  of  the  steps  taken  and  to  recognize  their  applicability 
to  the  design  of  any  defined  system.  Conversely,  the  strengths  of 
this  particular  demonstration  are  its  realism,  importance,  complexity, 
cost,  and  visibility. 

Hogarth  and  Makridakis  (1981b)  have  concluded  that  "values  and 
goals  are  difficult  to  communicate  and  often  must  be  sensed  by 
planners  rather  than  directly  obtained  from  policymakers . "  For  this 
reason,  construction  of  this  demonstration  was  primarily  achieved 
through  repeated  iteration  of  the  following  algorithm: 
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1)  Exploratory  discussions  were  held  between  the  researcher  and 
NASA  personnel. 

2)  The  researcher  synthesized  information  gained  from  the 
discussions  and  written  material  presented  by  NASA  into  a  structured 
representation  of  the  decision  problem. 

3)  The  output  of  step  2  was  then  used  as  a  "straw  man"  during 
follow-on  meetings  with  NASA  personnel.  Required  adjustments  were 
made  from  this  starting  point. 

It  was  felt  by  all  involved  that  this  approach  led  to  a  much  more 
complete  and  expedient  result. 


Since  the  focus  of  the  demonstration  concerns  the  implementation 
of  the  criterion  function  itself,  it  is  appropriate  to  point  out  key 
assumptions .  In  general ,  we  recognize  that  activities  prior  and 
subsequent  to  the  use  of  the  criterion  function  fall  largely  outside 
the  scope  of  this  research  and  are,  therefore,  assumed  to  be 
accomplished  completely  and  correctly.  In  particular,  we  make  the 
following  specific  assumptions  (Ostrofsky,  1977d:  331)  which  are 
consistent  with  previous  research  (Folkeson,  1982;  Peschke,  1986): 

1.  A  comprehensive  knowledge  of  the  candidate  systems  exists  to 
the  extent  that  the  resulting  criteria  measures  can  be  adequately 
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defined  in  terms  of  the  pertinent  design. 

2.  Those  who  are  evaluating  criterion  performance  and  its 
relative  value  are  experts  to  the  extent  that  their  decisions  are 
rational,  representing  the  maximum  level  of  accuracy  and  knowledge  at 
the  respective  stage  of  development. 

3.  The  decisions  formulated  from  the  resulting  criterion 
function  consider  only  the  criteria  identified  for  the  design  of  the 
system. 

4.  One  candidate  system  does  not  dominate  all  others  for  all 
criteria  so  that  a  logical  choice  of  the  optimal  system  from  the  set 
of  candidate  systems  is  not  apparent  without  further  analysis. 

5.  The  worth  of  the  emerging,  optimal  system  merits  the  effort 
involved  in  its  choice. 


B??kgrpund 


Subsequent  to  the  first  manned  lunar  landing  in  1969,  a 
presidential  task  group  recommended  that  the  United  States  develop  new 
technological  systems  for  space  operation  with  initial  activities 
directed  toward  the  achievement  of  a  new  space  transportation 
capability.  Toward  that  end,  in  1970  the  National  Aeronautics  and 
Space  Administration  (NASA)  initiated  engineering,  design,  and  cost 
studies  leading  to  the  development  of  the  Space  Transportation  System 
(STS),  the  prime  component  of  which  would  be  the  space  shuttle.  The 
final  design,  determined  in  1972,  called  for  a  space  shuttle  vehicle 
(orbiter)  with  an  external  fuel  tank,  augmented  by  two  reusable 
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external  solid  rocket  boosters  or  SRB's  (see  Figure  39).  (Note:  The 
facts  and  figures  cited  in  the  remainder  of  this  section  were 
primarily  obtained  from  NASA's  Space  Shuttle  News  Reference .  1989.) 

The  first  reusable  manned  space  flight  system,  the  shuttle  was 
designed  for  the  purpose  of  delivering  and  retrieving  a  wide  variety 
of  payloads  (e.g.  satellites,  experimental  platforms,  interplanetary 
vehicles,  etc.)  to  and  from  earth  orbit.  The  orbiter  is  approximately 
the  same  size  as  a  DC -9  aircraft,  measuring  121  feet  long,  57  feet 
high,  and  79  feet  wide.  It  weighs  approximately  150,000  pounds 
without  fuel  or  its  maximum  payload  of  approximately  65,000  pounds. 

A  typical  shuttle  mission  profile  calls  for  a  maximum  crew  of 
seven  astronauts  and  mission  specialists  to  launch  from  John  F. 
Kennedy  Space  Center  in  Florida  or  Vandenberg  Air  Force  Base , 
California.  Launch  is  accomplished  using  thrust  from  both  the 
orbiter 's  main  engines  and  the  SRB's.  The  external  fuel  tank  and  the 
SRB's  are  discarded  during  ascent.  Following  approximately  7  to  30 
days  of  operations  in  orbit,  the  crew  flies  the  orbiter  back  into  the 
earth's  atmosphere,  landing  at  Kennedy,  Vandenberg,  or  Edwards  Air 
Force  Base,  California. 

Currently,  the  orbiter  uses  hydraulic  power  to  activate  all 
flight  control  surfaces,  landing  gear,  brakes,  nose  wheel  steering, 
and  engine  controls.  To  provide  the  necessary  hydraulic  power,  the 
orbiter  employs  three  mechanical  power  generators  which  are  referred 
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to  as  Auxiliary  Power  Units  (APU's).  The  APU's  (Figure  40)  are  138- 
horsepower  turbine  engines  weighing  85  pounds  each  which  convert 
chemical  energy  from  liquid  hydrazine  fuel  into  mechanical  shaft 
power.  The  resulting  mechanical  power  is  converted  to  hydraulic  power 
via  variable -displacement,  constant  pressure  hydraulic  pumps.  In 
addition  to  the  engines,  fuel  cells,  and  hydraulic  pumps,  the  APU 
subsystem  includes  a  variety  of  supporting  hardware  such  as  fuel 
pumps,  valves,  and  a  water-spray  heat  exchanger  cooling  system.  The 
total  APU  system  weighs  a  combined  1860  pounds  and  occupies 
approximately  50  cubic  feet  of  space  at  the  rear  of  the  orbiter 
payload  bay. 

The  APU's  are  started  several  minutes  before  launch  and  are  shut 
down  approximately  when  orbit  is  achieved.  All  three  APU's  are 
required  for  this  ascent  phase.  The  APU's  are  reactivated  for  the 
descent  and  landing  phases;  however,  operation  of  any  two  of  the  three 
APU's  is  considered  sufficient  to  accomplish  these  activities. 

Beginning  with  the  initial  design  in  the  early  '70's,  NASA  and 
Rockwell  (the  prime  space  shuttle  contractor)  managers  and  engineers 
considered  the  possibility  of  alternate  APU  designs.  Concerns  with 
the  design  selected  and  currently  implemented  included  high  weight, 
excessive  servicing  requirements ,  and  safety  hazards  resulting  from 
the  use  of  hydrazine  fuel.  Renewed  interest  in  alternate  APU  designs 
emerged  in  the  early  '80's  with  the  advent  of  shuttle  operations 
(first  flight  in  April,  1981)  and  a  few  potentially  serious 
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Figure  40.  Auxiliary  Power  Units 
(NASA  Space  Shuttle  News  Reference,  1989) 


malfunctions  during  the  first  nine  flights  (1981-1983).  At  the  time 
of  this  research,  increased  interest  had  once  again  become  apparent  as 
evidenced  by  a  wide  variety  of  "in-house"  (NASA)  and  contractor 
studies  underway.  In  addition  to  resolution  of  the  concerns 
previously  identified,  a  significant  motive  for  reconsideration  of  APTJ 
design  is  the  facilitation  of  electro -mechanical  actuation  (EMA)  as  an 
alternative  to  hydraulic  power.  Essentially,  the  use  of  EMA  would 
involve  replacing  hydraulic  actuation  components  with  electric  motors. 
This  approach  is  appealing  for  variety  of  reasons,  the  foremost  of 
which  are  weight  reduction  and  substantial  improvements  in  the 
efficient  use  of  energy.  (EMA  systems  could  theoretically  result  in  a 
net  energy  use  of  near  zero  since  energy  imparted  to  move  control 
surfaces  is  returned  to  the  system  when  the  surfaces  return  to  neutral 
positions . ) 


Feasibility  Study 

Needs  Analysis 

At  the  outset  of  data  gathering  for  this  demonstration,  it  was 
perceived  that  the  problem  to  be  solved  concerned  the  broad  issue  of 
EMA  for  the  space  shuttle.  However,  subsequent  discussions  led  to  the 
realization  that  a  related,  but  essentially  independent  first  decision 
was  required  to  address  the  selection  of  a  design  for  a  new  APU  to 
facilitate  EMA.  While  the  use  of  EMA  was  being  considered  primarily 
as  a  single  general  approach,  the  new  APU  design  (typically  referred 
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to  as  electric  APU  or  EAPU)  was  known  to  require  selection  from  among 
a  variety  of  specifically  identified  known  or  theorized  component 
designs,  thereby  readily  lending  itself  to  the  type  of  analysis 
accommodated  by  the  criterion  function.  Therefore,  the  EAPU  was 
selected  as  the  subject  for  this  demonstration  as  a  first  step  toward 
EMA. 


Consequently,  the  "system"  being  designed  is  referred  to  as  the 
Auxiliary  Power  Unit  (APU).  As  described  earlier,  the  APU  system 
includes  mechanical  power  generators  which  are  used  to  drive  hydraulic 
pumps  which,  in  turn,  provide  hydraulic  power  for  the  following  space 
shuttle  flight  control  and  utility  systems: 

-  elevons 

-  rudder/speedbrake 

-  body  flap 

-  SSME  ATVC  (space  shuttle  main  engine  active  thrust 

vector  control) 

-  gear  deploy/uplock 

-  nose  wheel  steering 

-  brakes 

-  ME  (main  engine)  control 

-  umbilical  retract 

Briefings  and  discussions  with  NASA  managers  and  engineers 
produced  the  following  broad  statement  of  needs: 


Provide  a  means  for  space  shuttle  APU  power  generation 
which  provides  low  cost,  high  availability,  and  high  safety 


while  providing  performance  equal  to  or  greater  than  that  of 
the  current  system. 
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Identification  and  Formulation  of  the  Problem 


As  described  in  Chapter  2,  this  step  involves  specification  of 
desired  outputs,  undesired  outputs,  environmental  inputs,  and  intended 
inputs  for  each  of  the  Production/Consumption  Phases  of  the  system 
life  cycle.  The  following  definitions  were  assigned  to  the  APU  life 
cycle  phases: 


Production  Phase:  All  activities  involved  in  the  detailed  design, 
development,  planning,  documentation,  production,  and  procurement  of 
the  APU  system  and  its  planned  integration  into  the  space  shuttle 
system. 

Deployment  (Distribution)  Phase:  All  activities  concerned  with  the 
integration  of  the  APU  system  into  the  space  shuttle  system,  to 
include  hardware/software  modifications,  revision  of  space  shuttle 
documentation,  personnel  training,  and  resupply  (spares) . 

Operations  Phase:  The  use  and  support  of  the  APU  system  as  an 
integral  part  of  space  shuttle  operations . 

Retirement  Phase:  The  removal  from  operations  or  major  modification 
of  the  APU  system,  whether  concurrently  with  or  independently  from  the 
retirement  of  the  space  shuttle  system  Itself. 


Under  NASA's  approach  to  system  life  cycle  management,  the 
Production  Phase  and  portions  of  the  Deployment  Phase  would  be 
considered  part  of  DDT&E  (i.e.  Design,  Development,  Test  & 
Evaluation).  The  remainder  of  the  Deployment  Phase,  as  well  as  the 
Operations  and  Retirement  Phases  would  be  governed  via  Engineering 
Orders.  It  was  also  noted  that  NASA  training,  support,  and  spares 
considerations  are  typically  handled  through  separate  procurement 
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efforts  as  "spin-offs"  of  the  primary  design  activities.  Such  efforts 
are  initiated  based  on  any  perceived  impact  on  existing  or  planned 
training,  resupply,  or  datalink  download  (communication  of  in-flight 
data)  requirements . 

The  input-output  matrices  derived  for  this  demonstration  are 
shown  in  Tables  12-15.  The  recurring  themes  reflected  in  the  tables 
clearly  follow  the  stated  need:  availability  (including  reliability 
and  maintainability),  safety,  performance  (to  include  low  weight),  and 
cost. 


Synthesis  of  Solutions 

While  various  specific  design  alternatives  had  been  considered  or 
discussed  by  NASA  and  Rockwell  personnel,  no  evidence  was  found  of  an 
attempt  to  date  to  exhaustively  define  or  evaluate  a  set  of  candidate 
systems.  Therefore,  the  method  described  by  Ostrofsky  (1977d) ,  which 
may  be  described  as  the  strategy  generation  table  (Howard,  1988) ,  was 
employed . 

A  simplified  representation  of  the  current  APU  system  is  shown  in 
Figure  41.  One  alternative  concept  is  based  on  the  use  of  batteries 
as  an  energy  source  in  place  of  the  current  hydrazine  fuel  (Figure 
42).  Note  that  the  use  of  batteries  increases  cooling  requirements, 
quite  possibly  requiring  the  inclusion  of  a  new,  dedicated  cooling 
system.  A  second  new  concept  is  based  on  the  use  of  H2/O2  high  energy 
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TABLE  12. 
PRODUCTION  PHASE 


Desired  outputs: 

-  Reliable  components 

-  Maintainable  components 
•  Safe  components 

-  Compatible  components 

-  Quality  design,  materials,  and  fabrication 

-  Low  weight  system 


Undesired  outputs: 

-  Expensive  components 

-  Long  lead  times 

-  Unique  or  vulnerable  material,  production 

requirements 

-  High  risk 


Environmental  inputs: 

-  Specifications  for  existing  space  shuttle 

systems,  interfaces 

-  Contractor  design,  production  capabilities 

-  Existing  R&D  results 

-  Off-the-shelf  components 


Intended  inputs: 

-  New  R&D 

-  Funding 


TABLE  13. 


DEPLOYMENT  PHASE 


Desired  outputs: 

-  Efficient  and  timely  integration  with 

existing/planned  space  shuttle  systems 

Undesired  outputs: 

-  Adverse  impact  on  shuttle  operations 


Environmental  inputs : 

-  Scheduled  overhaul,  "down"  time  (modification 

period) 

-  Existing  contractor/NASA  facilities ,  tools , 

equipment 


Intended  inputs: 

-  Additional/new  facilities,  tools,  equipment 

-  Funding 

-  Qualified  engineers,  technicians,  etc.  to 

implement  changes 
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TABLE  14. 
OPERATIONS  PHASE 


Desired  outputs: 

-  Ease  of  checkout,  accessibility 

-  High  operational  availability 

-  High  level  of  safety  to  operators  and 

maintainers 

-  Low  cost  of  operations 

-  Valuable  information  for  future  applications 

-  Minimum  change  to  existing  operations 

( transparency) 

-  Visibility  of  system  performance 

-  Fail-op/fail-safe 
•  Good  performance 

-  Low  weight 

Undesired  outputs: 

-  Hazardous  material  utilization 

-  Unique  tools,  equipment 

-  High  impact  of  system  failures 


Environmental  inputs: 

-  Existing  space  shuttle  hardware/software  and 

interfaces 

-  Existing  support  capabilities  and  infrastructure 

(people,  facilities,  training,  tools,  etc.) 


Intended  inputs: 

-  Revised  operations/support  specifications 

-  Training  for  operators ,  maintainers 

-  Spares 

-  Funding 

-  Availability  of  consumables 
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TABLE  15. 
RETIREMENT  PHASE 


Desired  outputs: 

*  Inexpensive  deactivation 

-  Maximum  reusability  of  materials,  personnel, 

equipment 

-  Ease  of  overhaul/rebuild 

-  Retirement  consistent  with  spares  program 

-  Inputs  to  product  re-design,  improvement 

-  Efficient  disposition  of  retired  hardware 

-  Ease  of  change  implementation 


Undesired  outputs: 

-  Special  handling/storage  requirements 

-  Obsolescence 


Environmental  Inputs : 

-  NASA  plans  for  follow-on  systems,  operations 


Intended  inputs: 

-  Retirement  plans 

-  Funding 
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density  (HED)  fuel  cells  (Figure  43).  The  fuel  cell-based  concept 
more  closely  resembles  the  current  system  since  cooling  requirements 
are  normally  accommodated  via  self-contained  regulatory  systems  in  the 
fuel  cell,  avoiding  the  need  for  a  separate  dedicated  cooling  system. 
However,  the  use  of  HED  fuel  cells  avoids  the  need  for  toxic  fuels 
such  as  hydrazine.  Both  the  battery  and  HED  fuel  cell  concepts 
include  the  optional  use  of  variable  pressure  hydraulic  pumps .  While 
increasing  costs,  this  option  potentially  reduces  cooling  requirements 
and  provides  the  opportunity  for  substantial  weight  reductions. 

Specific  alternatives  had  been  identified  for  each  function  or 
subsystem  of  each  concept  with  the  exception  of  the  gearbox  and  the 
hydraulic  pumps.  While  differences  may  exist  in  alternatives  for 
these  two  components,  their  designs  were  deemed  sufficiently  similar 
to  make  any  differences  negligible  for  the  purposes  of  this  research 
relative  to  the  significant  differences  in  the  other  component 
designs.  The  exception  to  this  generalization  was  the  issue  of 
whether  or  not  to  pursue  the  variable  pressure  pump  option. 
Alternatives  considered  for  the  remaining  subsystems  were: 

Battery  energy  sources: 

-  Li-S02 

-  Li-SOCl2 

-  Ag-Zn  (Primary) 

-  Ag-Zn  (Secondary) 

-  Lead  Acid 

-  Ag-H2  (CPV) 

-  Ag-H2  (IPV) 


The  first  three  batteries  are  referred  to  as  primary  batteries, 
meaning  that  they  are  used  once  and  then  replaced.  The  last  four  are 
secondary  batteries,  meaning  that  they  can  be  re-charged  up  to  a 
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Figure  43 .  Fuel  Cell  System 
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certain  life  expectancy.  The  CPV  and  IPV  designations  for  Ag-H2 
batteries  stand  for  "common  pressure  vessel"  and  "individual  pressure 
vessel",  respectively,  indicating  whether  or  not  each  individual 
battery  cell  has  its  own  container.  While  the  former  is  somewhat 
cheaper  and  lighter,  the  latter  improves  safety  and  survivability  by 
increasing  containment  of  failures. 

Fuel  cell  energy  sources : 

-  Alkaline  electrolyte 

-  Solid  polymer  electrolyte  (SPE) 

-  Monolithic  solid  oxide 

-  Fuel  cell/battery  combination 

The  last  fuel  cell  alternative  involves  primary  use  of  fuel  cells  to 
handle  the  majority  of  power  demands  with  the  use  of  a  battery 
supplement  to  handle  the  portions  of  mission  profiles  which  call  for 
very  brief,  but  unusually  high  peaks  in  power  requirements. 

Cooling  systems: 

-  A  modified  version  of  the  existing  freon  system 

-  Bi -phase  wax 

-  Ammonia  boiler 

-  New  independent  system 

Motors : 

-  High-speed  induction 

-  Samarium  cobalt 

-  Switched  reluctance 

At  the  time  of  this  research,  preliminary  test  data  and  estimates 
were  available  for  many  characteristics  of  the  design  alternatives 
just  described.  However,  most  of  these  technologies  were  at  a 
sufficiently  early  stage  of  development  or  testing  that  the 
preliminary  estimates  were  considered  arithmetic  means  about  which 
different  implementations  would  vary.  Given  the  high  potential  value 
and  importance  of  obtaining  government  contracts  to  develop,  produce, 
and  support  these  technologies  it  was  expected  that  numerous 
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corporations  (hereafter  referred  to  as  contractors)  would  pursue  their 
development  if  and  when  sufficient  interest  is  expressed  by  NASA. 
NASA  personnel  perceive  that  the  differences  among  the  designs 
presented  by  the  various  contractors  would  be  sufficiently  important 
to  merit  their  explicit  consideration  in  the  decision  process. 

After  substantial  consideration  it  was  determined  that  a 
reasonable  approach  to  achieve  this  source  of  variation  was  through 
the  use  of  what  we  shall  call  contractor  or  design  profiles.  It  was 
anticipated  that,  in  general,  the  alternative  designs  for  the  APU 
energy  source,  cooling,  and  motor  subsystems  might  be  characterized  as 
management  oriented,  technically  oriented,  or  neutral.  Management 
oriented  designs  are  those  which  emerge  from  more  established, 
conservative  approaches  to  design.  Such  alternatives  may  be 
characterized  by  features  such  as  lower  cost  and  higher  reliability 
and  safety.  By  contrast,  technically  oriented  designs  are  those  which 
result  from  more  innovative,  state-of-the-art  approaches.  These 
designs  typically  trade  off  the  previously  mentioned  characteristics 
for  improved  technological  performance.  The  neutral  profile  is  an 
attempt  to  compromise  the  two  extremes.  Presumably,  the  more  specific 
the  guidance  NASA  provides  to  designers,  the  more  narrow  the  range  of 
this  source  of  variation.  However,  it  is  expected  that  sufficient 
leeway  would  always  exist  for  the  designing  organization  to  be  able  to 
orient  their  design  in  a  manner  which  reflects  their  own 
interpretations,  preferences,  and  strategies. 
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Clearly,  it  is  feasible  that  at  more  mature  stages  of 
development,  other  profiles  may  emerge.  For  example,  ideally,  one 
design  may  prove  superior  in  all  regards.  However,  should  this  occur, 
sophisticated  analysis  becomes  unnecessary.  The  issue,  then,  is  which 
design  is  best  if  a  dominant  alternative  does  not  emerge.  Therefore, 
the  three  design  profiles  described  were  explicitly  considered  for 
each  power  source,  cooling,  and  motor  alternative.  Essentially,  then, 
the  mean  characteristic  estimates  were  represented  as  the  neutral 
profile  and  were  augme.  ted  with  alternatives  which  displayed  each  of 
the  other  two  specified  design  orientations. 

Synthesis  of  candidate  systems  was  then  accomplished  as  suggested 
by  Ostrofsky  (1977d)  by  forming  all  possible  combinations  of  subsystem 
alternatives  for  each  concept,  resulting  in  4,753  candidate  systems  as 
follows : 

Current  system:  1  candidate  system 

Battery-powered  systems:  7  battery  options  *  3  battery 
design  profiles  *  4  cooling  system  options  *  3 
cooling  design  profiles  *  3  motor  options  *  3  motor 
design  profiles  *  2  variable  pump  options 

-  4,536  candidate  systems 

Fuel  cell-powered  systems:  4  fuel  cell  options  *  3  fuel 
cell  design  profiles  *  3  motor  options  *  3  motor 
design  profiles  *  2  variable  pump  options 

-  216  candidate  systems 

Total  -  1  +  4,536  +  216  -  4,753  candidate  systems 

Screening  of  Candidate  Systems 


Since  the  synthesis  of  candidate  systems  is  accomplished  from  a 
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relatively  unconstrained  point  of  view,  this  step  is  intended  to 
enhance  the  ability  of  the  analyst  to  evaluate  the  set  of  candidates 
by  eliminating  those  which  clearly  cannot  or  should  not  be  considered 
further.  The  three  specific  screens  recommended  by  Ostrofsky  (1977d) 
are  those  of  physical  realizability,  economic  worthwhileness,  and 
financial  feasibility. 

With  respect  to  physical  realizability,  NASA  personnel 
acknowledged  that  future  testing  and  evaluation  might  reveal  some  of 
the  candidate  systems  to  be  unachievable.  In  particular,  it  was 
unclear  whether  the  cooling  requirements  of  some  battery  powered 
candidates  could  be  accommodated  solely  by  modifications  to  the 
existing  freon  system.  However,  no  data  was  available  which 
conclusively  eliminated  the  possibility.  Therefore,  the  principle  of 
least  commitment  was  invoked  and  none  of  the  candidate  systems  were 
eliminated. 

With  respect  to  economic  worthwhileness,  opinions  differed 
slightly  among  NASA  personnel.  Proponents  of  each  concept  or  specific 
alternative  tended  to  be  somewhat  more  pessimistic  about  the 
worthwhileness  of  other  candidates.  However,  all  candidate  systems 
appeared  to  have  supporters.  In  addition,  the  determination  of 
worthwhileness  in  state-of-the-art  design  efforts  such  as  this  is 
confounded  by  the  possibility  of  spin-off  applications  which  might 
enhance  the  worth  of  an  otherwise  inferior  alternative.  Since  all 
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candidates  identified  were  perceived  worthwhile  both  for  the  intended 
purpose  and  for  possible  future  applications,  all  were  retained. 

Finally,  the  determination  of  financial  feasibility  is  typically 
based  on  budget  or  other  resource  constraints.  In  this  case,  no 
overall  budget  has  been  established  since  the  current  and  previous 
studies  have  been  exploratory  in  nature.  At  this  stage,  the 
evaluation  effort  is  proceeding  one  step  at  a  time  with  each  funding 
decision  based  on  both  resource  availability  and  the  perceived 
importance  or  merit  of  the  resulting  design.  Consequently,  NASA 
personnel  were  unable  to  state  unequivocally  that  any  of  the 
candidates  identified  would  necessarily  be  financially  infeasible. 
Again,  the  principle  of  least  commitment  mandated  that  the  defined  set 
of  candidate  systems  remain  intact  based  on  the  information  currently 
available . 


Preliminary  Activities 

Preparation  for  Analysis 

This  step  in  the  design  morphology  provides  at  least  two 
significant  opportunities.  The  first  is  to  establish  a  period  of 
review  and  evaluation  which  is  particularly  helpful  in  linking  the 
activities  of  the  Feasibility  Study  to  those  of  the  Preliminary 
Activities  when  some  period  of  time  has  elapsed  between  the  two.  In 
this  demonstration,  no  such  time  lapse  occurred.  The  second 
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significant  opportunity  is  that  of  gaining  insight  into  the  set  of 
candidate  systems  by  examining  possible  groupings  and  evaluating  each 
group's  apparent  advantages  and  disadvantages.  The  following 
paragraphs  briefly  review  the  most  significant  insights  perceived  by 
NASA  personnel  and  the  author. 

The  strongest  perceived  grouping  of  the  candidate  systems  was 
that  reflected  in  the  two  major  concepts:  delineation  by  energy 
source.  Throughout  the  analysis  it  was  apparent  that  NASA  perceived 
the  system  weight  to  be  a  crucially  important  performance 
characteristic.  The  logic  of  this  concern  was  quite  apparent  since 
the  purpose  of  the  shuttle  is  to  transport  payload,  making  weight  an 
important  constraint  on  the  extent  to  which  its  mission  can  be 
accomplished.  Both  concepts  were  generally  expected  to  produce  weight 
reductions.  However,  fuel  cell-powered  candidates  were  expected  to 
have  a  slight  weight  advantage,  primarily  due  to  the  reduced  cooling 
requirements.  Fuel  cell-powered  alternatives  also  offer  the 
advantages  of  R&D  synergism  with  the  Strategic  Defense  Initiative 
(SDI)  and  good  safety  characteristics,  but  are  based  on  less  mature 
technology  and  are  expected  to  be  somewhat  more  complex,  with  greater 
aircrew  operations  and  groundcrew  servicing  requirements  than  battery- 
powered  alternatives .  Another  tradeoff  is  that  the  replacement  of 
failed  fuel  cells  would  be  much  more  expensive  than  battery 
replacement . 

Among  battery  types,  secondary  batteries,  while  costing  more  than 


primary  batteries,  offer  the  advantages  of  increased  life  expectancy, 
a  more  mature  stage  of  development,  and  generally  improved  safety 
characteristics . 

Another  significant  tradeoff,  mentioned  earlier,  is  that 
candidates  with  the  variable  pressure  pump  option  were  expected  to 
cost  more  but  produce  substantial  weight  reductions. 

Advantages  and  disadvantages  based  on  other  groupings  appeared 
less  clear-cut  and  more  difficult  to  generalize. 

Definition  of  Criteria 


As  the  discussion  thus  far  indicates,  the  choice  of  criteria  was 

relatively  straightforward.  Beginning  with  the  needs  analysis  and 

reinforced  through  the  identification  and  formulation  of  the  problem, 

four  predominant  decision  criteria  emerged: 

x^  -  Cost 
X2  -  Performance 
X3  -  Availability 
X4  -  Safety 

By  forming  all  possible  combinations,  these  four  marginal 

criteria  lead  to  the  synthesis  of  11  possible  criteria  intersections: 

x^2  "  Cost/Performance 

x^3  -  Cost/Availability 

x14  '  Cost/Safety 

x23  ’  Performance/Availability 

x24  ‘  Performance/Safety 
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X34  ■  Availability/Safety 

x123  ‘  Cost/Performance/Availability 

x124  ‘  Cost/Performance/Safety 

x134  "  Cost/Availability/Safety 

x234  *  Performance/Availability/Safety 

x1234  "  Cost/Performance/Availability/Safety 

Relative  Weights 

Perhaps  the  most  enlightening  step  accomplished  in  the 

demonstration  was  that  of  specifying  relative  weights  for  the 

criteria.  Two  characteristics  of  the  weighting  process  were  quite 
novel  to  the  NASA  personnel,  eventually  proving  to  be  quite  exciting 
and  relevant  to  their  decision  process. 

First  was  the  possibility  of  relative  weights  varying  throughout 
the  possible  range  of  criteria  performance  (X^) .  Initially,  it  was 
not  apparent  to  NASA  personnel  how  this  could  be  the  case.  However, 
during  extended  discussions  it  became  apparent  to  both  decision  makers 
and  the  author  that  their  basis  for  expressing  preferences  did, 
indeed,  depend  on  the  quality  of  candidate  system  performance.  For 
example,  cost  (x^)  only  became  substantially  important  as  the  cost  of 
the  system  increased.  As  long  as  the  system's  cost  was  relatively 
good  (low) ,  their  preferences  were  primarily  based  on  the  other 
criteria.  However,  as  cost  increased  (performance  on  x^  decreased), 
their  concern  for  this  criterion  increased  exponentially.  In 
addition,  changes  in  relative  importance  were  not  always  found  to  be 
monotonically  increasing  or  decreasing  throughout  the  range.  The 
importance  of  availability  (X3)  for  example  was  found  to  be 


overshadowed  by  that  of  other  criteria  at  both  extremes  of  the  range 
so  that  it  reached  its  zenith  in  the  middle  of  the  range. 
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The  second  important  characteristic  of  the  weighting  process 
concerned  the  weighting  of  criteria  intersections.  Initially,  the 
meaning  of  intersection  relative  weights  proved  quite  elusive  to  the 
decision  makers.  Therefore,  the  author  was  led  to  innovate  new 
approaches  to  describing  and  eliciting  these  weights.  The  vehicle 
which  proved  most  useful  in  this  regard  was  a  simple  2X2  matrix  with 
the  name  of  one  criterion  in  each  cell.  Presented  with  this  matrix, 
NASA  personnel  were  first  asked  to  circle  that  cell  or  combination  of 
cells  which  best  described  the  most  important  basis  (or  decision  rule) 
they  would  use  in  selecting  from  among  candidate  systems.  Clearly, 
the  natural  first  tendency  might  be  to  circle  all  four  cells. 
Therefore,  it  was  important  to  point  out  that  a  preference  for  good 
performance  on  multiple  criteria  might  (and  probably  would)  incur 
sacrifices  on  performance  of  subsets  of  that  combination.  In  other 
words,  by  preferring  a  combination  of  cost  and  performance  they  would 
be  indicating  a  willingness  to  sacrifice  some  measure  of  either  cost 
or  performance  in  order  to  achieve  a  better  combination  or  balance  of 
the  two. 

The  simultaneous  consideration  of  both  of  these  characteristics 
(variance  throughout  the  range  and  consideration  of  intersections) 
required  repeated  iterations  through  the  weighting  process. 
Ultimately,  a  high  degree  of  consistency  was  obtained  at  which  point 
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both  the  decision  makers  and  the  author  felt  that  the  derived  relative 
weights  (Figures  44-50)  represented  the  true  preference  structure  with 
a  high  degree  of  accuracy.  Various  test  comparisons  of  hypothetical 
candidate  systems  supported  the  validity  of  these  relative  weights. 

Note  that  of  the  eleven  possible  criteria  intersections,  three 
fir*,  t-order  intersections  (xjj,  X23,  and  X24)  were  identified  as 
having  significant  importance:  the  others  were  not  considered 
important.  However,  it  is  also  interesting  to  note  that  two  of  the 
intersections  (Performance/Availability  and  Performance/Safety)  were 
deemed  to  be  generally  more  important  than  the  marginal  criteria 
through  much  of  the  range. 

The  relative  weights  shown  in  Figures  44-50  reflect  the  following 
functional  relationships : 


-  f  1  if  XjSI.01 


L  | (log  Xi)/2 |  if  Xi>.01 

Eq.  5.1 

a2  -  f  .5  if  XjS.5 

\  Xi  if  Xj> . 5 

Eq.  5.2 

a3  -  2Xi-2Xi2 

Eq.  5.3 

a4  -  r  1-Xi  if  X^.5 

\  .5  if  Xj>.5 

Eq.  5.4 

a13  -  r  .3  if  XiSs.5 

\  .e-oxi/s)  if  Xi>. 5 

Eq.  5.5 

a23  “  -7 

Eq.  5.6 
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Figure  44.  Cost  Relative  Weight 
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Figure  48.  Cost/Availability  Relative  Weight 
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a24 


1-Xi  if  X^.5 
Xj_  if  Xj>.  5 


All  other  a^-0 . 


Eq.  5.7 


Although  many  of  these  functions  contain  inflection  points ,  none 
involve  discontinuities .  Therefore ,  the  use  of  Model  VII  is 
indicated. 


Definition  of  Parameters 

In  order  to  obtain  figures  of  merit  for  the  criteria  identified 
(and  their  intersections)  it  was  necessary  to  specify  the  constituent 
elements  of  each  to  a  level  of  detail  which  would  facilitate  some  form 
of  measurement.  Numerous  iterations  throughout  the  course  of  the 
analysis  produced  the  criterion  elements  shown  in  Tables  16-19.  Each 
element  will  be  explained  in  the  section  on  Criterion  Modeling. 

As  described  in  Chapter  2,  elements  were  coded  as  "a"  if  they 
could  be  directly  measured  via  some  (preferably  objective)  scale. 
These  elements  are  hereafter  referred  to  as  parameters  and  are  denoted 
as  yjj.  A  total  of  41  parameters  emerged  for  this  demonstration. 
Thirty-four  of  these  parameters  were  deemed  measurable  using  objective 
units.  Standard  subjective  scales  were  established  for  the  remaining 
seven  parameters. 


Twenty-six  elements  in  Tables  16-19  were  coded  "b". 


These  will 
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TABLE  16. 

COST  ELEMENTS 

Criterion  xjj  Cost 

Element  Code 


Expected  initial  cost  b 
Initial  cost  estimate  b 
Gross  acquisition  cost  a 
Risk  factor  b 
Maturity  of  technology  a 
Contractor  risk  b 
Battery  contractor  success  a 
Battery  contractor  attempts  a 
Fuel  cell  contractor  successes  a 
Fuel  cell  contractor  attempts  a 
Motor  contractor  successes  a 
Motor  contractor  attempts  a 
Cooling  system  contractor  successes  a 
Cooling  system  contractor  attempts  a 
Recurring  costs  b 
Replacement  costs  b 
Number  of  replacement  parts  required  b 
Number  of  missions  planned  a 
APU  operating  hours  per  mission  a 
System  life  expectancy  a 
Unit  cost  to  replace  b 
Cost  of  new/refurbished  unit  a 
Serial  hours  for  replacement  a 
Parallel  hours  for  replacement  a 
Cost  of  serial  hours  a 
Cost  of  parallel  hours  a 
Servicing  costs  b 
Number  of  servicing  actions  required  b 
Time  between  servicing  a 
Cost  of  servicing  action  b 
Quantity  of  consumables  for  servicing  a 
Cost  of  consumables  a 
Serial  hours  for  servicing  a 
Parallel  hours  for  servicing  a 
Long  term  costs  b 
Disposal/retirement  costs  a 
Savings  due  to  future  applications/ 

extensions  a 


TABLE  17. 

PERFORMANCE  ELEMENTS 


Criterion  X£:  Performance 

Element  Code 

Physical  requirements  b 

Weight  a 

Volume  a 

Operating  temperature  range  a 

Impact  on  operations  b 

Crew  operations  requirements  a 

Complexity  a 

Integration  effects  b 


Impact  on  performance  of  other  systems  a 

Ability  to  facilitate  beneficial 
changes 


a 


TABLE  18. 


AVAILABILITY  ELEMENTS 

Criterion  X3:  Availability 

Element  Code 

Initial  availability  b 

Acquisition  time  a 

Installation  time  a 

Risk  factor  b 

Operational  availability  b 

Total  possessed  calendar  hours  b 

Mission  length  a 

Time  between  missions  a 

Down  time  hours  b 

Maintenance  down  time  b 

System  life  expectancy  a 

Time  to  replace  b 

Serial  hours  for  replacement  a 

Parallel  hours  for  replacement  a 

Time  between  servicing  a 

Time  to  service  b 

Serial  hours  for  servicing  a 

Parallel  hours  for  servicing  a 

Mean  operating  time  between  maintenance  b 
Probability  of  mission  completion  b 

Number  of  APU  systems  installed  a 

Number  of  APU's  for  mission  completion  a 


TABLE  19. 


SAFETY  ELEMENTS 


Criterion  x^:  Safety 

Element  Code 

Flight  safety  b 

Tolerance  to  abuse  a 

Potential  severity  of  failure  a 

Probability  of  mission  completion  b 

Ground  safety  b 

Hazardous  handling  requirements  a 

Maintenance  down  time  b 


Mean  operating  time  between  maintenance  b 
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be  referred  to  as  submodels  (or  simply  models)  and  denoted  as  zj .  The 
next  section  describes  each  submodel,  its  constituent  parameters,  and 
the  parameter  units  of  measure. 

Subscripts  (1-41)  were  assigned  sequentially  to  the  parameters  in 
the  order  shown  in  Table  20.  However,  since  more  than  one  "layer"  of 
submodels  was  frequently  necessary  to  relate  parameters  to  criteria, 
subscripts  for  submodels  were  assigned  to  reflect  the  hierarchical 
relationship  between  the  submodel  and  its  parent  criterion.  For 
example,  Z32  is  the  second  first- level  submodel  used  to  compute 
criterion  X3.  Similarly,  Z321  is  the  first  submodel  used  in  the 
computation  of  Z32. 

As  a  means  of  helping  to  insure  consistency,  completeness,  and 
compactness  of  the  parameter  set,  Ostrofsky  (1977d)  has  suggested  the 
construction  of  a  table  relating  parameters  to  criteria  through 
submodels.  Unfortunately ,  the  large  number  of  parameters  and 
submodels  used  in  this  demonstration  makes  the  depiction  of  such  a 
table  problematic.  Therefore,  two  such  tables  have  been  constructed. 
The  first.  Table  20,  relates  parameters  directly  to  the  highest  level 
submodel  for  each  criterion.  This  table  provides  a  useful  vehicle  for 
identifying  missing,  unnecessary,  or  ambiguous  parameters.  The  second 
table.  Table  21,  relates  all  submodels  to  the  criteria  in  which  they 
are  used. 


At  this  point,  it  is  appropriate  to  point  out  that  a  complete  set 
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TABLE  20. 

RELATION  OF  PARAMETERS  TO  SUBMODELS 


*1  X2  X3  X4 


£11  Z12  £13  £21  £22  £23  £31  £32  £33  £41  £42 


Parameters 

1  Gross  acq  cost  X 

2  Maturity  of  tech  X 

3  Battery  successes  X 

4  Battery  attempts  X 

5  Fuel  cell  successes  X 

6  Fuel  cell  attempts  X 

7  Motor  successes  X 

8  Motor  attempts  X 

9  Cooling  successes  X 

10  Cooling  attempts  X 

11  #  missions  planned  X 

12  APU  OH  per  mission  X 

13  System  life  expect  X 

14  Cost  of  new  unit  X 

15  Serial  hrs  replace  X 

16  Parallel  hrs  replace  X 

17  Cost  of  serial  hrs  X 

18  Cost  of  parallel  hrs  X 

19  Time  between  service  X 

20  Qty  of  consumables  X 

21  Cost  of  consumables  X 

22  Serial  hrs  service  X 

23  Parallel  hrs  service  X 

24  Dispose/retire  costs  X 

25  Future  appl/ext  X 

26  Weight 

27  Volume 

28  Ops  temp  range 

29  Crew  ops  reqmts 

30  Complexity 

31  Impact  other  systems 

32  Facilitate  good  chgs 

33  Acquisition  time 

34  Installation  time 

35  Mission  length 

36  Time  betw  missions 

37  #  APU's  installed 

38  #  APU's  required 

39  Tolerance  to  abuse 

40  Severity  of  failure 

41  Hazardous  handling 


X 

X 

X 

X 

X 

X 

X 

X 

X 


X  X  X  X 

X 
X 


X 


X 

X 


X 

X 

X 

X 

X 

X 

X 

X 

X 

XXX 

X 

X  X 
X  X 

X  X 
X  X 
X 


XX  X  XX 


TABLE  21. 


RELATION  OF  SUBMODELS  TO  CRITERIA 


X1  x2  x3  x4 


Submodels 


£11  £12  £13  &21  £22  £23  £31  £32  £33  £41  £42 


Expected  initial  cost  X 
Initial  cost  estimate  X 
Risk  factor  X 

Contractor  risk  X 

Recurring  costs  X 

Replacement  costs  X 

#  replacements  required  X 

Unit  cost  to  replace  X 

Servicing  costs  X 

#  svcing  actions  reqd  X 

Cost  of  svcing  action  X 

Long  term  costs  X 

Physical  requirements  X 

Impact  on  operations  X 


Integration  effects 
Initial  availability 
Ops  availability 
Total  possessed  hrs 
Down  time  hours 
Maint  down  time 
Time  to  replace 
Time  to  service 
Mean  time  between  maint 
Prob  mission  completion 
Flight  safety 
Ground  safety 


X 

X 


X 


X 


X 

X 

X 

X 

X 

X 

X 


X 


X 

X 

X 


X 
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of  parameter  values  did  not  exist  for  the  set  of  4,753  candidate 
systems.  While  such  data  was  clearly  desirable,  its  a  priori 
existence  would  be  an  unreasonable  expectation  since,  as  stated 
previously,  the  complete  set  of  candidate  systems  was  not  synthesized 
prior  to  this  research.  Furthermore,  it  should  be  remembered  that 
many  of  the  subsystems  considered  currently  exist  in  theory  or  early 
prototype  form  only.  Therefore,  it  was  necessary  to  combine  the  known 
data  and  estimates  in  as  realistic  a  manner  as  possible  pending  the 
ability  to  verify  such  estimates  through  subsequent  testing.  This  is 
a  common  characteristic  of  high  technology  projects.  In  fact,  Dr. 
Aaron  Cohen,  Director  of  Johnson  Space  Center,  has  observed  (1989) 
that  the  first  principle  of  maximizing  success  in  project  management 
is  that  "you  must  fearlessly  base  your  decisions  on  the  best 
information  available."  To  facilitate  these  estimates,  the  method 
referred  to  by  Tversky  and  Kahneman  (1974)  as  the  "anchoring  and 
adjustment  heuristic"  was  employed.  This  method  is  accomplished  by 
first  identifying  a  known  reference  point  ("anchor")  and  then 
adjusting  for  the  effects  (both  marginal  and  interactive)  of  the  other 
relevant  factors.  By  having  used  this  approach,  we  are  not  only  able 
to  proceed  with  the  analysis  based  on  the  best  existing  information, 
but  have  also  identified  those  data  requirements  which  should  serve  as 
the  basis  for  subsequent  test  and  evaluation  activities. 

Criterion  Modeling 


The  following  paragraphs  describe  the  way  in  which  measurable 


parameter  values  were  combined  for  this  demonstration  to  produce 
meaningful  figures  of  merit  for  each  criterion.  In  some  cases, 
validated  theoretical  or  logical  models  are  available  or  apparent  to 
serve  as  a  guide.  In  other  cases,  the  analyst  must  determine  a 
meaningful  method  of  combination.  In  such  cases,  the  key  is  to 
produce  a  metric  which  responds  to  candidate  system  differences  in  a 
manner  which  unambiguously  reflects  better  or  worse  performance,  even 
though  the  resulting  units  of  measure  themselves  may  not  display  any 
apparent  meaning.  To  insure  consistency,  the  models  for  this 
demonstration  were  constructed  in  such  a  way  that,  even  if  lower 
submodel  values  were  good,  models  at  the  criterion  level  would  insure 
that  higher  raw  criteria  scores  always  reflected  better  performance 
than  low  criteria  scores . 

Before  introducing  the  submodels,  it  is  appropriate  to  cite  the 
following  caveats.  First,  it  is  recognized  that,  as  with  any  modeling 
effort,  the  models  could  be  made  more  detailed.  Particularly  when 
dealing  with  complex  systems,  there  is  virtually  no  end  to  the  depth 
to  which  models  can  be  formulated.  In  addition,  it  is  acknowledged 
that  the  models  used  might  be  made  more  rigorous.  For  example,  many 
of  the  metrics  formed  here  by  taking  simple  products  might  have 
theoretical  alternatives.  However,  both  of  these  considerations  must 
be  traded  off  against  both  the  accuracy  of  the  data  and  the  objectives 
of  the  research.  In  this  regard,  the  models  presented  and  used  were 
deemed  appropriate  to  adequately  assess  the  relative  performance  of 
the  candidate  systems  and,  more  importantly  for  the  purposes  of  this 
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research,  to  facilitate  the  demonstration  of  the  proposed  method  for 
criterion  function  computation. 


Xj  -  Cost 

This  model  reflects  cost  performance  by  taking  the  inverse  of  the 
sum  of  net  primary  life  cycle  costs.  The  constituent  elements  are: 
z^i  -  Expected  initial  cost 
7.yi  -  Recurring  costs 
2^3  -  Long  term  costs 

All  values  were  converted  to  millions  of  dollars  prior  to  inclusion  in 
this  model.  The  expression  used  to  compute  x^  is: 

xi“V(zil+zi2+z13)  Eq.  5-8 

zn  -  Expected  Initial  Cost 

This  model  reflects  the  anticipated  initial  costs  of  development 
and  implementation.  Experienced  NASA  personnel  recognize  that  not  all 
initial  estimates  prove  to  be  equally  valid.  Therefore,  the  model 
explicitly  includes  a  risk  factor  which  inflates  the  initial  estimate 
in  a  manner  which  reflects  the  contractor's  (developer/producer) 
"track  record"  in  previous  design  efforts.  The  constituent  elements 
are: 

-  Initial  cost  estimate  (Eq.  5.10) 
z112  ‘  Risk  factor  (Eq.  5.11) 

The  resulting  expression  is: 


zll“zlll*z112 


Eq.  5.9 
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Zjn  -  Initial  cost  estimate 

Normally,  this  submodel  would  include  the  various  inputs  to 
initial  implementation  costs,  e.g.  R&D,  production,  raw  materials, 
direct  labor,  burden,  etc.  However,  in  prior  EAPU  studies,  total 
gross  initial  cost  estimates  had  been  provided  to  NASA  by  Rockwell 
without  the  underlying  breakout.  Therefore,  while  it  was  deemed 
prudent  to  explicitly  identify  the  existence  of  a  submodel,  the 
expression  used  for  this  demonstration  was  simply: 

zui-yi  Eq-  5.io 

where  y^  is  the  gross  acquisition  cost  measured  in  millions  of 
dollars . 

z112  -  Rlsk  factor 

As  mentioned  previously,  this  submodel  attempts  to  reflect  the 

inflationary  tendency  typically  experienced  with  risky  alternatives. 

The  constituent  elements  are: 

y2  -  Maturity  of  technology 
(0  -  new/untested, 

1  -  laboratory  testing, 

2  -  off-the-shelf) 

Zll2i  *  Contractor  risk  (Eq.  5.12) 

The  expression  for  *s: 


*112-l/«(y2+2)/4)*z1121) 


Eq.  5.11 
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z1121  -  Contractor  risk 


This  submodel  reflects  the  degree  to  which  the  prospective 

subsystem  contractor  (developer/producer)  has  succeeded  in  meeting 

comparable  previous  obligations.  The  constituent  parameters  are: 

y3  -  Battery  contractor  successes  (The  number  of  comparable 
contract  deliverables  or  milestones  the  contractor  has 
achieved  on  time  and  successfully  meeting 
specifications) 

y^  -  Battery  contractor  attempts  (The  number  of  comparable 
contract  deliverables  or  milestones  the  contractor  has 
undertaken) 

y5  -  Fuel  cell  contractor  successes 
yg  -  Fuel  cell  contractor  attempts 
y 7  -  Motor  contractor  successes 
yg  -  Motor  contractor  attempts 
yg  -  Cooling  system  contractor  successes 
y^Q  -  Cooling  system  contractor  attempts 
The  submodel  is  computed  as : 

i/y4  Vy6 

zii2i“(<y3/y4)  )*<(y5/y6>  > 

i/y8  i/yio 

*((y7/ys)  )*«y9/yio>  )  Eq.  5.12 


This  model  identifies  the  expected  operational  costs  due  to 
replacement  and  servicing.  The  constituent  submodels  are: 
z121  "  Replacement  costs  (Eq.  5.14) 
z122  ’  Servicing  costs  (Eq.  5.17) 


The  expression  is: 


z12"z121+z122 


Eq.  5.13 


z121  -  Replacement  costs 

This  model  computes  the  expected  costs  of  replacement  due  to 
failure .  It  includes : 

z1211  "  Number  of  replacements  required  (Eq.  5.15) 
z1212  *  Unit  cost  to  replace  (Eq.  5.16) 

It  is  computed  as: 

z121”z1211*z1212  EcI-  5-14 

z1211  ~  Number  of  replacements  required 


This  submodel  estimates  the  expected  number  of  times  the  system 
will  need  to  be  replaced  over  the  life  of  the  shuttle  program.  It 
considers : 

-  Number  of  missions  planned  (This  parameter  is  a 
constant.  Original  NASA  plans  were  targeted  for 
100  missions  for  each  of  4  vehicles.  However,  the 
unexpectedly  slow  growth  in  the  shuttle  sortie  rate 
would  seem  to  make  that  figure  optimistic,  even  with 
the  possibility  of  delayed  shuttle  retirement. 

Therefore,  an  estimate  of  300  was  used.) 

yi2  -  APU  operating  hours  per  mission  (Although  each  mission 
may  range  from  7  to  30  days,  the  flight  profile  calls 
for  APU  operation  only  during  ascent,  descent,  and 
landing.  A  constant  of  1.5  operating  hours  was  used.) 

yi3  -  System  life  expectancy  (operating  hours) 

The  submodel  is: 


zi2ii-<yn*yi2>/yi3 


Eq.  5.15 


This  submodel  includes  both  the  hardware  replacement  and  repair 

time  costs.  The  constituent  parameters  are: 

-  Cost  of  new/refurbished  unit  (thousands  of  dollars) 

yi5  -  Serial  hours  for  replacement  (Repair  hours  during 

which  no  other  work  can  be  simultaneously  accomplished 
on  the  orbiter.) 

y^g  -  Parallel  hours  for  replacement  (Repair  hours  during 
which  other  orbiter  repairs  or  servicing  can  also  be 
be  accomplished  in  parallel.) 

yi7  -  Cost  of  serial  hours  (Estimated  at  $28,700  per  hour) 
y^g  -  Cost  of  parallel  hours  ($19,000  per  hour) 

The  submodel  is : 

zi2i2-yi4+(yi5*yi7>+(yi6*yi8>  E<i-  5-16 

&122-  ~  Servicing 

This  model  computes  the  expected  costs  due  to  required  routine 
servicing.  The  logic  of  the  model  and  its  submodels  parallels  that  of 
z121-  Constituent  submodels  for  servicing  costs  are: 

z1221  ‘  Number  of  servicing  actions  required  (Eq.  5.18) 
z1222  '  Cost  of  servicing  action  (Eq.  5.19) 

The  submodel  is : 


z122“z1221*z1222 


Eq.  5.17 
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z1221  -  Number  of  servicing  actions  required 


This  submodel  computes  the  expected  number  of  times  the  APU  will 

require  servicing.  It  is  based  on: 

yil  -  Number  of  missions  planned  (300) 

yi2  -  APU  operating  hours  per  mission  (1.5) 

y^9  -  Time  between  servicing  (All  of  the  candidates 

identified  required  some  form  of  servicing  after  every 
flight.  Therefore,  this  parameter  was  treated  as  a 
constant  of  1.5  operating  hours.) 

The  submodel  is : 


zi22i-(m*yi2>/yi9  e<j-  5-18 

(Note:  It  is  recognized  that,  for  the  set  of  candidate  systems 

identified  in  this  demonstration,  z^221  wi-H  always  be  300.) 


£1222  =-£gg&-gf  servicing  actfop 


The  constituent  parameters  are: 

yu  -  Cost  of  serial  hours  ($28,700) 

yi8  -  Cost  of  parallel  hours  ($19,000) 

y2o  *  Quantity  of  consumables  for  servicing  (lbs.) 

y2i  -  Cost  of  consumables  ($/lb.) 

y22  -  Serial  hours  for  servicing  (hours) 

y23  -  Parallel  hours  for  servicing  (hours) 

The  model  is : 


zi222“<y2o*y2i>+<y22*yi7>+(y23*yi8> 


Eq.  5.19 


This  submodel  computes  the  expected  net  cost  of  longer  term 

considerations.  It  includes: 

y24  *  Disposal/retirement  costs  (millions  of  dollars) 

y25  -  Savings  due  to  future  applications/extensions 

(Specific  opportunities  identified  included  EMA, 
use  on  solid  rocket  boosters,  and  SD1.  Measured  in 
millions  of  dollars.) 

The  expression  is: 

zi3“y24-y25  £q-  5-20 


xo  -  Performance 

This  criterion  considers  a  variety  of  relevant  technical 
performance  measures .  It  should  be  noted  that  other  relevant 
performance  measures  which  might  normally  be  included  in  the 
evaluation  of  a  new  APU  design  may  not  be  included  here  since  they  are 
constrained  by  the  requirements  of  the  existing  space  shuttle  and 
would  therefore  be  included  as  required  specifications.  Since  the 
measures  used  here  reflect  less  desirable  performance  as  the  units 
increase,  the  criterion  measure  takes  the  inverse  of  their  product. 
The  pertinent  elements  are: 

Z21  "  Physical  requirements  (Eq.  5.22) 

Z22  ‘  Impact  on  operations  (Eq.  5.23) 

Z23  *  Integration  effects  (Eq.  5.24) 

The  expression  is: 


x2-l/(z2i*z22*z23> 


Eq.  5.21 


248 


Z21  -  Physical  requirements 


This  submodel  conglomerates  the  more  important  physical  features 
of  the  system.  Its  elements  are: 
y26  -  Weight  (lbs.) 
y27  -  Volume  (cubic  feet) 

y28  -  Operating  temperature  range  (degrees  C) 

The  submodel  is: 

z2i~<(y26)3*y27)/y28  Ecl-  5  22 

Note  that  y28  has  been  raised  to  the  third  power  to  reflect  its 

perceived  significance  relative  to  the  other  two  parameters. 


zoo  -  Impact  on  operations 


It  is  desirable  that  a  new  system  be  as  transparent  and/or  simple 
to  operate  and  support  as  possible  both  on  the  ground  and  in  flight. 
This  submodel  attempts  to  assess  the  magnitude  of  potential  impacts  in 
terms  of: 

y29  -  Crew  operations  requirements  (Number  of  minutes  of 
direct  crew  attention  or  manipulation  required  in 
flight.) 

y30  -  Complexity 

(1  -  Very  simple  for  both  ground  and  flight  operations 

2  -  Somewhat  complex  either  in  flight  or  on  ground 

3  -  Somewhat  complex  both  in  flight  and  on  ground 

4  -  Very  complex  either  in  flight  or  on  ground 

5  -  Very  complex  both  in  flight  and  on  ground) 

The  model  is: 


z22“y29*y30 


Eq.  5.23 
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223  -  Integration  effects 


This  model  explicitly  recognizes  that  integration  of  a  new  APU 

system  into  the  existing  orbiter  may  have  good  and/or  bad  effects  on 

other  orbiter  systems.  The  model  therefore  considers: 

y31  -  Adverse  impact  on  performance  of  other  systems 
(1  -  Little  or  no  impact  on  other  systems 

2  -  Moderate  impact  on  other  systems 

3  -  Significant  impact  on  other  systems) 

y32  -  Ability  to  facilitate  beneficial  changes 

(1  -  Does  nothing  to  facilitate  beneficial  changes 

2  -  Potentially  minor  beneficial  changes  facilitated 

3  -  Potentially  major  beneficial  changes  facilitated) 

The  submodel  is: 


z23“,y3l/y32  Eq.  5.24 


X3  -  Availability 


This  criterion  simultaneously  considers  three  different  measures 
of  the  extent  to  which  the  system  will  be  operational  when  needed. 
The  constituent  submodels  are: 

Z31  -  Initial  availability  (Eq.  5.26) 

Z32  -  Operation  availability  (Eq.  5.27) 

Z33  -  Probability  of  mission  completion  (Eq.  5.35) 

The  expression  is: 


x3"(z32*z33)/z31 


Eq.  5.25 
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z3i  -  Initial  availability 

This  submodel  assesses  how  soon  the  system  could  be  made 
available  for  operational  use.  Since,  like  initial  costs,  this 
estimate  typically  proves  to  be  understated,  it  is  modified  by  the 
risk  factor  identified  previously.  The  model  considers: 
y33  -  Acquisition  time  (years) 
y34  -  Installation  time  (weeks) 
z112  *  Risk  factor  (Eq.  5.11) 

The  submodel  is: 

z31“(y33+(y34/52)  )*z112  Eq-  5  26 

(Note:  To  avoid  division  by  zero  in  Eq.  5.25,  the  results  of  this 

submodel  were  arbitrarily  inflated  by  one  year.  This  modification 
simply  facilitated  computation  and  had  no  effect  on  the  relative 
performance  of  the  candidate  systems  since  it  added  the  same  constant 
to  all  candidates . ) 

Z32  -  Operational  availability 

Essentially,  operational  availability  measures  the  portion  of 
time  a  system  will  be  available  for  use.  This  model  and  its 
associated  submodels  follow  Seger's  (1989)  revision  of  the  traditional 
computation  (Blanchard,  1986)  in  order  to  accommodate  the  difference 
between  systems  such  as  the  APU,  which  are  operated  intermittently, 
and  those  which  are  operated  continuously.  It  utilizes: 
z321  *  Total  possessed  calendar  hours  (Eq.  5.28) 
z322  "  Down  time  hours  (Eq.  5.29) 
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The  model  Is: 

z32"(z321'z322)/z321  EtI-  5-27 

Z321  -  Total  possessed  calendar  hours 

This  model  simply  determines  the  duration  of  an  operating  cycle 
(the  time  from  the  start  of  one  use  until  the  start  of  the  next  use). 
It  includes: 

y35  -  Mission  length  (operating  hours) 

y36  -  Time  between  missions  (A  mean  estimate  of  6  months 
was  selected  as  a  compromise  between  current  and 
intended  usage . ) 

The  model  is  the  sum  of  the  two: 

z321”y35+(y36*30*24)  E<1-  5  28 


£322—  EggP-tlffiS  hfflIS 

This  model  computes  the  expected  length  of  down  time  incurred  by 
servicing  and/or  replacement  per  operating  cycle.  It  is  based  on: 
y35  -  Mission  length  (operating  hours) 

Z3221  •  Maintenance  down  time  (Eq.  5.30) 

Z3222  '  Mean  operating  time  between  maintenance  (Eq.  5.33) 

The  submodel  is : 


z322”(y35*z3221>/z3222 


Eq.  5.29 
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z3221  -  Maintenance  down  time 

This  submodel  follows  Blanchard's  (1986)  method  for  determining 
the  average  portion  of  the  time  the  system  will  be  unavailable  due  to 
replacement  and/or  servicing  actions .  The  constituent  elements  are : 
yL3  -  System  life  expectancy  (operating  hours) 

232211  ”  Time  to  replace  (Eq.  5.31) 

y^9  -  Time  between  servicing  (operating  hours) 

Z32212  ‘  Time  to  service  (Eq.  5.32) 

The  model  is  computed  as : 

z3221“^z3221l/yi3)+^z32212/yi9))/((1/yi3'>4'(1/yi9))  E9-  5-30 

z32211  ~  Time  to  replace 

This  submodel  reflects  the  total  time  for  a  replacement  action. 
It  consists  of: 

yi5  -  Serial  hours  for  replacement  (hours) 
y^g  -  Parallel  hours  for  replacement  (hours) 

The  model  is  simply: 

z32211“yi5+yi6  Ecl-  5-31 

Z32212  -  Time  to  service 


Similarly,  this  submodel  totals  the  time  required  for  a  servicing 
action.  The  relevant  times  are: 

y22  -  Serial  hours  for  servicing 
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y23  *  Parallel  hours  for  servicing 
The  submodel  is: 

z32212“y22+y23  E<i-  5-32 

23222  -  Mean  operating  time  between  maintenance 

This  submodel  follows  Blanchard  (1986)  based  on: 
yi3  -  System  life  expectancy  (operating  hours) 
y^9  -  Time  between  servicing  (operating  hours) 

It  is  computed  as: 

z3222“l/((l/yi3>+(l/yi9))  Eq.  5.33 

^-----grobftbiuty  of  mission  <?pigpi?pign 

Whereas  operational  availability  simply  assesses  an  overall  mean 
percentage  of  (essentially  calendar)  availability,  this  submodel 
attempts  to  assess  the  potential  that  the  AFU's  may  fail  to  operate  to 
the  extent  required  on  each  specific  mission.  It  follows  the 
frequently  used  model  presented  by  Blanchard  (1986).  Experience 
indicates  that  the  time  between  failures  for  electro -mechanical 
devices  typically  approximates  a  Weibull  distribution.  The  form  of 
the  Weibull  distribution  is  described  as  follows  (Besterfield,  1979) : 

0-1  -a(t-y)P 

f(t)  -  a0(t-7)  e  Eq.  5.34 

With  the  exception  of  expected  means,  the  preliminary  nature  of  the 
estimates  used  in  this  demonstration  precluded  the  availability  of 
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empirical  distributions  which  would  be  useful  in  providing  meaningful 
estimates  of  the  remaining  Weibull  parameters .  Therefore ,  to 
facilitate  comparison  of  candidate  systems,  it  was  arbitrarily  assumed 
that  /3-1  and  7-O,  in  which  case  the  Weibull  reduces  to  the  exponential 
distribution.  (Consequently,  the  failure  rate  is  assumed  Poisson.) 


Submodel  Z33  uses: 

yi3  -  System  life  expectancy  (operating  hours) 
y35  -  Mission  length  (operating  hours) 


y37  -  Number  of  APU  systems  installed  (3) 

y3g  -  Number  of  APU  systems  required  for  mission 
completion  (2) 

The  model  is  computed  as: 


y37-y38  x  -((y37*y35>/yi3> 

((y37*y35)/y13)  *e 

z33  -  2  . 

x! 

x-0 


Eq.  5.35 


This  criterion  considers  the  perceived  level  of  safety  resulting 
from  the  design  in  terms  of  both  flight  and  ground  operations.  The 
constituent  submodels  are: 

Z41  *  Flight  safety  (Eq.  5.37) 

Z42  -  Ground  safety  (Eq.  5.38) 

Safety  is  computed  as : 


x4"z41*z42 


Eq.  5.36 
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z/,i  -  Flight  Safety 


This  sumbmodel  treats  flight  safety  as  a  function  of  the 
likelihood  of  failure  as  well  as  its  potential  consequences.  It 
includes : 

y39  -  Tolerance  to  abuse 

(1  -  Fragile,  easily  damaged 

2  -  Reasonably  tolerant  if  treated  as  designed 

3  -  Highly  tolerant,  even  if  abused) 

y40  -  Potential  severity  of  failure 

(1  -  Consequences  of  failure  are  not  severe 

2  -  Failure  may  incapacitate  components  but  is  not 

expected  to  threaten  life  or  mission 

3  -  Failure  may  incapacitate  entire  systems, 

threatening  life  and/or  mission  completion) 

Z33  -  Probability  of  mission  completion  (Eq.  5.35) 

The  model  is: 


z41“z  3 3*y 3  9/y40  Eq .  5.37 


z/,2  -  Ground  safety 


This  model  computes  ground  safety  as  a  function  of  the  risks 

faced  by  maintenance  technicians  as  well  as  the  frequency  and  duration 

of  exposure  to  those  risks.  The  model  considers: 

y39  -  Tolerance  to  abuse  (described  for  z4^) 

y40  -  Potential  severity  of  failure  (described  for  z4^) 

y4l  -  Hazardous  handling  requirements 
(1  -  Little  or  none 

2  -  Low  hazard  infrequently  required 

3  -  Low  hazard  frequently  required 

4  -  High  hazard  infrequently  required 

5  -  High  hazard  frequently  required) 

Z3221  •  Maintenance  down  time  (Eq.  5.30) 
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z3222  "  Mean  operating  time  between  maintenance  (Eq.  5.33) 
The  submodel  is: 


z42“(z3222*y39)/(Z3221*y40*y4l)  E<1-  5-38 


Formulation  of  the  Criterion  Function 


Since  the  criteria  relative  weights  are  represented  as  continuous 
functions  and  since  the  need  exists  to  consider  intersection  terms, 
Model  VII  (as  described  in  Chapters  2  and  4)  provides  the  appropriate 
form  of  the  criterion  function: 


CF  - 


n  n  n 

i-i  i  j  J  J 

i/j 


n  n  n 

+  2  2  2  Sijk^ijk 

i/j 

j/k 

i/k 


Eq.  5.39 


n  n 

-  ...  +22 
i  j 
i/j 


n 


...  2  S 
J+l 


ijk.. .(J+l) 


ijk...(J+l) 


J/J+l 


where 

S i?i  -  5^A^F(x^)  Eq.  5.40 


The  F(x^)-X^,  representing  criteria  performance,  and  are  computed  via 
the  models  in  Eq.  5.8-5.38  and  Eq.  4.46.  The  (relative  weights) 


are  obtained  as  described  in  Chapter  4,  beginning  with  the  functions 
aj-f^CX^)  as  defined  in  Eq.  5. 1-5. 7  or  a^-0  otherwise. 
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Chapter  4  included  a  discussion  of  the  use  of  empirical 
distribution  functions  as  opposed  to  continuous  approximations.  Based 
on  the  reasoning  presented  in  that  discussion,  the  empirically 
observed  distributions  were  used  for  the  computations  in  this 
demonstration.  Additional  factors  supporting  their  use  were  the 
sufficiently  large  number  of  candidate  systems  (well  in  excess  of  the 
desired  500  minimum)  and  the  fact  that  computation  using  empirical 
distributions  has  not  been  demonstrated  in  previous  criterion  function 
research. 

When  continuous  c.d.f.  approximations  are  employed  it  is 
necessary  to  determine  minima  and  maxima  for  the  x^  (and  therefore  for 
the  yk  and  Zj  as  well)  to  facilitate  curve  fitting.  With  the  use  of 
empirical  distributions  this  is  no  longer  necessary  since  computation 
is  accomplished  via  Eq.  4.46  (Parzen,  1960:  25)  based  on  the  observed 
step  function  (Feller,  1966:  36).  However,  it  is  recognized  that 
additional  insight  may  be  gained  via  these  determinations.  Therefore, 
the  minima  and  maxima  for  this  demonstration  are  presented  in  Tables 
22  (yk) ,  23  (zj),  and  24  (xt) . 


The  next  step  called  for  in  the  design  morphology  is  the  Analysis 
of  the  Parameter  (or  Design)  Space.  In  an  actual  implementation  where 
design  decisions  will  continuously  be  modified  during  the  life  of  the 
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TABLE  22. 
PARAMETER  RANGES 


Parameters 

Minimum 

Maximum 

1  Gross  acquisition  cost 

0 

165 

2  Maturity  of  technology 

0 

2 

3  Battery  contractor  successes 

1 

8 

4  Battery  contractor  attempts 

1 

8 

5  Fuel  cell  contractor  successes 

1 

8 

6  Fuel  cell  contractor  attempts 

1 

8 

7  Motor  contractor  successes 

1 

8 

8  Motor  contractor  attempts 

1 

8 

9  Cooling  system  contractor  successes 

1 

8 

10  Cooling  system  contractor  attempts 

1 

8 

11  Number  of  missions  planned 

300 

300 

12  APU  operating  hours  per  mission 

1.5 

1.5 

13  System  life  expectancy 

1.275 

17,250 

14  Cost  of  new/refurbished  unit 

40 

19,800 

15  Serial  hours  for  replacement 

6.8 

27.6 

16  Parallel  hours  for  replacement 

6.8 

46 

17  Cost  of  serial  hours 

28.7 

28.7 

18  Cost  of  parallel  hours 

19 

19 

19  Time  between  servicing 

1.5 

1.5 

20  Quantity  of  consumables  for  servicing 

0 

325 

21  Cost  of  consumables 

10 

50 

22  Serial  hours  for  servicing 

6.8 

52.9 

23  Parallel  hours  for  servicing 

26.35 

104.48 

24  Disposal/retirement  costs 

0 

10 

25  Savings  due  to  future  applications/ext 

0 

34.5 

26  Weight 

624.5 

5638.13 

27  Volume 

38.25 

66 

28  Operating  temperature  range 

80 

120 

29  Crew  operating  requirements 

10 

15 

30  Complexity 

1 

5 

31  Impact  on  performance  of  other  systems 

1 

3 

32  Ability  to  facilitate  beneficial  changes 

1 

3 

33  Acquisition  time 

0 

5.75 

34  Installation  time 

0 

17 

35  Mission  length 

1.5 

1.5 

36  Time  between  missions 

6 

6 

37  Number  of  APU  systems  installed 

3 

3 

38  Number  of  APU  systems  for  mission  completion  2 

2 

39  Tolerance  to  abuse 

1 

3 

40  Potential  severity  of  failure 

1 

3 

41  Hazardous  handling  requirements 

1 

5 

TABLE  23. 


SUBMODEL  RANGES 


Submodels 

Minimum 

Z11 

Expected  initial  cost 

0 

zlll 

Initial  cost  estimate 

0 

z112 

Risk  factor 

1 

z1121 

Contractor  risk 

25 

z12 

Recurring  costs 

2  o.75 

z121 

Replacement  costs 

.0095 

z1211 

#  replacements  required 

.026 

z1212 

Unit  cost  to  replace 

364.36 

z122 

Servicing  costs 

208.74 

z1221 

#  svcing  actions  reqd 

300 

z1222 

Cost  of  svcing  action 

695.81 

z13 

Long  term  costs 

-34.5 

Z21 

Physical  requirements 

7 . 8xl07 

z22 

Impact  on  operations 

10 

z23 

Integration  effects 

.33 

Z31 

Initial  availability 

1 

z32 

Ops  availability 

.9723 

z321 

Total  possessed  hrs 

4321.5 

z322 

Down  time  hours 

22.58 

z3221 

Maint  down  time 

22.58 

z32211 

Time  to  replace 

13.6 

z32212 

Time  to  service 

33.15 

z3222 

Mean  time  between  maint 

.6892 

z33 

Prob  mission  completion 

.1327 

Z41 

Flight  safety 

.0442 

z42 

Ground  safety 

.00139 

Maximum 

1320 

165 

8 

1 

8632.16 

7576.28 

352.94 

21,466.12 

1055.88 

300 

3519.6 

10 

1.48xlOn 

45 

3 

49.62 

.9948 

4321.5 

119.74 

55.02 

73.6 

157.38 

1.5 

.9999 

2.9997 

.19929 
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TABLE  24. 
CRITERION  RANGES 


Criterion 
x^  Cost 
X2  Performance 
X3  Availability 
X4  Safety 


Minimum 

.0001004 

5.005xl0'14 

.0026 

.0000614 


Maximum 
.0057389 
3 . 885xl0'9 
.9947 
.5978102 
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project  based  on  test  results  and  other  new  information,  this  step 
provides  invaluable  feedback  concerning  how  adjustments  in  design 


parameters  should  be  made.  When  accomplished  thoroughly,  the 
activities  of  sensitivity  and  stability  analysis  can  be  expected  to 
require  large-scale  programming  and  computational  effort.  If  NASA 
ultimately  decides  to  pursue  the  use  of  the  morphology  to  assist  their 
design  process,  this  step  would  become  essential.  However,  such 
efforts  were  judged  to  exceed  the  scope  of  the  present  demonstration 
since  this  research  proposes  no  changes  in  the  method  of  their 
accomplishment.  Therefore,  the  description  proceeds  with  formal 
optimization. 


Formal  Optimization 

All  computational  work  for  this  demonstration  was  accomplished  on 
personal  computers.  In  addition  to  facilitating  the  convenience  of 
the  author,  avoidance  of  mainframe  computing  was  desired  to  support 
the  contention  that  the  methods  proposed  can  be  easily  implemented  by 
prospective  users,  even  in  the  absence  of  sophisticated  hardware.  The 
vast  majority  of  computing  was  accomplished  on  the  author's  Zenith  Z- 
171  portable.  For  the  longest  running  step  (conversion  of  raw 
criteria  scores  to  X^)  ,  a  Compaq  Model  II  portable  with  an  80286 
processor  was  used  to  reduce  the  required  run  time. 

Three  separate  programs  were  written  to  evaluate  the  defined  set 
of  candidate  systems  in  accordance  with  the  proposed  method  described 
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in  Chapter  4.  The  programs  were  initially  written  and  tested  using 
interpreted  BASIC.  They  were  subsequently  compiled  and  executed  using 
TURBO  BASIC  in  order  to  improve  run  times  and  provide  sufficiently 
large  variable  storage  capabilities.  The  process  flow  showing  the 
relationship  among  the  programs  is  depicted  in  Figure  51. 

The  first  program,  LITTLE-X.BAS ,  converted  candidate  system 
parameter  values  to  raw  criteria  scores  using  the  submodels  previously 
described  in  this  chapter.  A  flowchart  of  the  program  is  shown  in 
Figure  52  and  a  complete  source  code  listing  is  contained  in  Appendix 
B.  Given  the  large  number  of  candidate  systems,  parameter  values  were 
initially  stored  in  six  data  files.  The  program  simply  read  in  the 
parameter  values  for  a  candidate  system,  computed  the  submodels  in  Eq. 
5.8-5.38  and  stored  the  resulting  raw  criteria  scores  in  an  output 
file .  This  process  was  repeated  until  scores  were  computed  for  all 
candidate  systems.  The  large  number  of  candidate  systems  required  the 
use  of  two  such  output  files. 

The  second  program,  BIG-X.BAS,  converted  the  raw  criteria  scores 
(x^)  produced  by  LITTLE-X.BAS  to  normalized  scores  (X^) ,  including 
intersection  terms  ) .  As  mentioned  previously,  this  was 
accomplished  via  a  simple  counting  algorithm  based  on  Eq.  4.46  and  the 
empirical  distribution  of  raw  criteria  scores .  A  flowchart  of  the 
program  is  given  in  Figure  53  and  the  program  source  listing  is 
included  as  Appendix  C.  Since  this  program  required  the  longest  run 
time,  two  versions  were  run  in  parallel,  each  performing  computations 
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Figure  51.  Overall  Computation  Flowchart 


to  Normalized  Scores  and  Intersections 
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for  half  of  the  candidate  systems,  but  both  necessarily  based  on  the 
distribution  of  the  complete  set  of  candidates.  Each  program  produced 
one  output  file. 


The  final  program,  CF.BAS,  read  each  candidate's  values  (as 
output  from  BIG-X.BAS),  determined  the  appropriate  relative  weights, 
normalized  the  weights,  and  wrote  the  resulting  CF  value  to  a  single 
output  file.  The  program  also  output  the  best  and  worst  candidate 
systems  to  the  screen  along  with  their  CF  values.  The  logical  flow  of 
this  program  is  shown  in  Figure  54  and  the  source  listing  is  contained 
in  Appendix  D. 

It  should  be  noted  that  none  of  the  program  source  code  is 
particularly  complex  or  lengthy.  Furthermore,  the  programs  may  be 
easily  adapted  to  solution  of  other  four-criteria  problems  by  simply 
changing : 

a.  The  input  and  output  files  used  by  each  program, 

b.  The  number  of  candidate  systems  to  be  evaluated  in  each 
program, 

c.  The  submodels  in  LITTLE -X. BAS ,  and 

d.  The  relative  weights  in  CF.BAS. 

Only  slightly  more  involved  modifications  would  be  required  to 
adapt  the  programs  to  cases  of  other  than  four  criteria.  Even  these 
modifications  would  be  primarily  programmatic  in  nature  (more  or  fewer 
variables  and  loops)  with  the  logical  flow  remaining  unchanged. 


Figure  54.  Computation  of  CF  Values 
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This  section  has  described  the  mechanics  of  the  optimization 
process.  The  final  section  of  this  chapter  reviews  and  discusses  the 


specific  results  obtained  for  this  demonstration. 

Discussion  of  Results 

Results  described  in  this  section  were  obtained  using  MINITAB, 
LOTUS  1-2-3,  and  a  variety  of  ad  hoc  BASIC  programs  run  against  the 
various  program  output  files. 

Candidate  system  CF  values  ranged  from  a  high  of  .9326  to  a  low 
of  .3119.  The  optimal  candidate  system  (CF-.9326)  consisted  of  Ag-H2 
IFV  batteries  with  a  neutral  design  profile  as  the  power  source,  a  new 
independent  cooling  system  (with  a  management -oriented  design) , 
switched- reluctance  motors  (also  with  a  management -oriented  design), 
and  variable  pressure  pumps .  The  parameter  values  for  the  optimal 
candidate  system  are  shown  in  Table  25.  This  system  achieved 
scores  greater  than  .75  on  all  four  marginal  criteria  and  greater  than 
.5  on  all  intersection  terms.  Interestingly,  it  was  also  the  best  of 
all  candidate  systems  with  respect  to  criterion  X4,  safety. 

At  this  point  in  the  analysis,  it  is  appropriate  to  consider  the 
effect  of  the  accuracy  of  the  results  on  the  choice  of  the  optimal 
candidate.  Specifically,  accuracy  limitations  on  data  estimates, 
computations,  and  curve  fitting  may  all  serve  to  moderate  apparent 


TABLE  25 


OPTIMAL  CANDIDATE  PARAMETER  VALUES 

Parameter  Value 

1  Gross  acquisition  cost  138.53 

2  Maturity  of  technology  1 

3  Battery  contractor  successes  3 

4  Battery  contractor  attempts  4 

5  Fuel  cell  contractor  successes  1 

6  Fuel  cell  contractor  attempts  1 

7  Motor  contractor  successes  8 

8  Motor  contractor  attempts  8 

9  Cooling  system  contractor  successes  8 

10  Cooling  system  contractor  attempts  8 

11  Number  of  missions  planned  300 

12  APU  operating  hours  per  mission  1.5 

13  System  life  expectancy  150 

14  Cost  of  new/refurbished  unit  108.24 

15  Serial  hours  for  replacement  8 

16  Parallel  hours  for  replacement  8 

17  Cost  of  serial  hours  28.7 

18  Cost  of  parallel  hours  19 

19  Time  between  servicing  1 . 5 

20  Quantity  of  consumables  for  servicing  0 

21  Cost  of  consumables  0 

22  Serial  hours  for  servicing  8 

23  Parallel  hours  for  servicing  31 

24  Disposal/retirement  costs  0 

25  Savings  due  to  future  applications/ext  10 

26  Weight  1626.37 

27  Volume  50 

28  Operating  temperature  range  95 

29  Crew  operating  requirements  10 

30  Complexity  1 

31  Impact  on  performance  of  other  systems  1 

32  Ability  to  facilitate  beneficial  changes  3 

33  Acquisition  time  3.2 

34  Installation  time  17 

35  Mission  length  1.5 

36  Time  between  missions  6 

37  #  of  APU  systems  installed  3 

38  #  of  APU  systems  for  mission  completion  2 

39  Tolerance  to  abuse  3 

40  Potential  severity  of  failure  1 

41  Hazardous  handling  requirements  3 
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differences  among  CF  scores .  In  other  words ,  a  small  difference  in  CF 
scores  may,  in  fact,  be  insignificant  given  the  level  of  accuracy  of 
the  inputs  to  those  results.  In  this  demonstration,  the  accuracy  of 
the  original  sources  of  estimation  was  generally  unknown.  Therefore, 
to  facilitate  comparison,  it  was  assumed  that  the  accuracy  of  the 
estimates  was  adequate  to  support  all  subsequent  computations .  In 
practice,  perceptions  of  distinctions  among  candidates  should  not  be 
based  on  differences  which  are  finer  than  the  accuracy  of  the 
constituent  sources  of  measurement.  Instead,  candidates  whose 
differences  are  not  greater  than  the  accuracy  of  their  measurement 
should  be  treated  as  indistinguishable.  In  such  cases,  it  may  be  most 
useful  to  pay  more  attention  to  groups  of  candidates  formed  by 
significant  breaks  in  the  ranking  of  CF  scores. 

The  worst  candidate  system  (CF-.3119)  consisted  of  Li-S0Cl2 
batteries,  the  existing  freon  cooling  system,  samarium  cobalt  motors 
(all  with  technically- oriented  design  profiles),  and  constant  pressure 
pumps.  This  candidate's  highest  X^  score  (X2)  was  .4551  and  its 
remaining  X^  scores  were  all  less  than  .1. 

The  current  APU  system  achieved  a  CF  score  of  .4828.  A  review  of 
its  scores  indicated  that  the  only  reason  it  was  able  to  obtain  a 
score  as  high  as  it  did  is  that  it  achieved  the  top  rating  in 
availability  (X4)  .  This  is  explained  by  the  fact  that  its  initial 
availability  (the  system  is  already  in  use)  overwhelmed  the  remaining 
candidate  systems ,  all  of  which  would  require  approximately  two  years 
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or  more  to  implement.  With  the  exception  of  X4,  the  current  system 
was  able  to  achieve  no  higher  than  .2729  for  any  criterion. 

Recognizing  that  the  results  for  any  single  candidate  system  may 

be  anomalous,  Table  26  is  presented  to  help  support  generalizations. 

To  facilitate  representation  in  the  available  space,  the  candidate 

systems  are  coded  in  the  following  mai ,  e” 

1st  position  -  Battery/fuel  cell  energy  .ource 

2nd  position  -  Energy  source  design  profile 

3rd  position  -  Cooling  system  (for  battery-powered  systems) 

4th  position  -  Cooling  system  design  profile 
5th  position  -  Motor  type 
6th  position  -  Motor  design  profile 
7th  position  -  Hydraulic  pump  option 

The  values  used  to  represent  alternatives  in  the  first,  third,  and 

fifth  positions  were  assigned  sequentially  in  the  order  the 

alternatives  were  presented  earlier  in  this  chapter.  The  management- 

oriented,  technically-oriented,  and  neutral  design  profiles  are 

denoted  as  1,  2,  and  3,  respectively.  A  "1"  in  the  seventh  position 

would  indicate  constant  pressure  pumps  while  a  "2"  indicates  variable 

pressure  pumps . 

Examination  of  Tsble  26  suggests  the  following  insights.  First, 
the  dominant  single  characteristic  is  the  preference  for  variable 
pressure  hydraulic  pumps:  it  is  the  sole  characteristic  common  to  all 
50  top  performers.  The  second  most  clearly  evident  preference 
concerns  the  use  of  Ag-H2  IPV  batteries  as  the  APU  energy  source. 
This  characteristic  is  common  to  47  of  the  top  50  candidates,  with  the 
next  best  energy  source  (Ag-Zn  secondary  batteries)  not  emerging  until 
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NOTE 


TABLE  26 

TOP  50  CANDIDATE  SYSTEMS 


Rank  Candidate  CF  Rank  Candidate  CF 


1 

7341312 

0.9326 

2 

7341332 

0.9314 

3 

7343312 

0.9303 

A 

7321312 

0.9287 

5 

7321332 

0.9271 

6 

7343332 

0.9270 

7 

7323312 

0.9262 

8 

7323332 

0.9238 

9 

7341112 

0.9068 

10 

7341322 

0.9063 

11 

7341132 

0.9060 

12 

7343112 

0.9049 

13 

7141332 

0.9042 

14 

7143312 

0.9038 

15 

7141312 

0.9031 

16 

7321322 

0.9030 

17 

7143332 

0.9027 

18 

7341212 

0.9025 

19 

7343132 

0.9024 

20 

7321112 

0.9013 

21 

7343212 

0.9005 

22 

7241312 

0.9004 

23 

7341232 

0.9004 

24 

7321132 

0.8999 

25 

7323112 

0.8989 

26 

7321212 

0.8988 

27 

7121332 

0.8982 

28 

7342312 

0.8981 

29 

7123312 

0.8980 

30 

7121312 

0.8972 

31 

7123332 

0.8969 

32 

7221312 

0.8968 

33 

7321232 

0.8968 

34 

7343232 

0.8966 

35 

7323212 

0.8964 

36 

7323132 

0.8964 

37 

7322312 

0.8963 

38 

7311312 

0.8955 

39 

7323232 

0.8926 

40 

7311332 

0.8919 

41 

7241332 

0.8905 

42 

7313312 

0.8905 

43 

7141322 

0.8901 

44 

7142312 

0.8870 

45 

4341312 

0.8868 

46 

4341332 

0.8867 

47 

7243312 

0.8865 

48 

7221332 

0.8860 

49 

7343322 

0.8855 

50 

4343312 

0.8855 

The  top-rated  fuel  cell  candidate  was:  2100312,  CF  -  .7953 
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the  45th  position.  As  the  note  at  the  bottom  of  the  table  indicates, 
the  top-rated  fuel  cell-powered  candidate  (using  solid  polymer 
electrolyte)  failed  to  make  the  top  50,  but  scored  a  respectable 
.7953. 

Perhaps  the  next  most  prominent  observation  emerging  from  the 
table  is  an  apparent,  although  less  pronounced,  preference  for  the  use 
of  switched-reluctance  motors.  In  fact,  the  most  distinct  break  in 
the  table's  CF  scores  occurs  between  the  eight  (CF-.9238)  and  ninth 
(CF-.9068)  candidates  when  switched-reluctance  motors  are  first 
replaced  with  an  alternative  motor.  Prior  to  that  point  in  the 
rankings,  the  combination  of  Ag-H2  IPV  batteries,  switched-reluctance 
motors,  and  variable  pressure  pumps  were  the  only  three 
characteristics  common  to  all  eight  candidates. 

Finally,  it  is  apparent  that  management -oriented  and  neutral 
design  profiles  were  generally  preferred  to  technically- oriented 
profiles.  This  would  seem  to  support  the  choice  of  more  proven, 
conservative  subsystem  designs  over  state-of-the-art  but  riskier 
alternatives.  This  apparent  preference  may  perhaps  reflect  the  fact 
that  although  the  decision  makers  participating  in  this  demonstration 
are  highly  qualified  in  terms  of  technical  knowledge,  their  duties  and 
responsibilities  might  be  described  as  primarily  managerial  in  nature. 
Other  possible  explanations  might  include  a  general  risk- averse 
orientation  on  the  part  of  either  the  individual  decision  makers  or 
the  organization  as  a  whole . 


Chapter  6 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

Summary 

The  topic  of  this  dissertation  has  been  decision  making  with 
multiple,  potentially  interacting  criteria  and  varying  relative 
weights  within  the  context  of  system  design.  The  term  "system"  has 
been  broadly  defined  as  "any  set  of  objects  and/or  attributes  and  the 
relationships  among  them."  The  term  "design"  has  been  used 
synonymously  with  "planning"  and  has  been  used  to  refer  to  a  series  of 
purposeful  activities  directed  toward  the  achievement  of  a  particular 
system. 

Chapter  1  presented  a  review  of  significant  developments  in 
system  design.  The  primary  developments  cited  were  the  formalization 
of  distinguishable  design  phases,  explicit  consideration  of  the 
complete  set  of  system  life  cycle  phases,  the  significant  influence  of 
systems  theory,  and  the  iterative  nature  of  design.  Chief  among  the 
specific  influences  of  systems  theory  were  the  focus  on  a  macro  (vs. 
micro)  level  of  design,  an  expansionist  (as  opposed  to  reductionist) 
perspective,  and  a  deductive  (as  opposed  to  inductive)  approach  to 
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problem  solution.  The  remainder  of  Chapter  1  delineated  desirable 
characteristics  of  methods  for  design  decision  making.  This 
discussion  provided  the  foundation  for  the  research  goals  by  citing 
the  need  for  a  prescriptive  decision  model  for  complex,  multi-criteria 
design  decision  problems.  Specific  desirable  characteristics  cited 
included  the  explicit  consideration  of  both  objective  and  subjective 
factors,  the  accommodation  of  relative  weights  which  vary  throughout 
the  range  of  criteria  performance,  the  objective  of  optimizing  (rather 
than  simply  satisficing)  criteria,  and  the  ability  to  accommodate 
interaction  among  criteria. 

Chapter  2  described  a  specific  design/planning  methodology  which 
served  as  a  framework  for  the  balance  of  the  research.  In  the 
Feasibility  Study,  the  problem  is  generally  defined  and  then  further 
detailed  for  each  of  the  system  life  cycle  phases.  Alternative 
solutions  (candidate  systems)  are  then  synthesized  and  subsequently 
screened  to  ensure  their  suitability  for  further  analysis.  In  the 
Preliminary  Activities,  broad  decision  criteria  are  defined  and 
relative  weights  assigned.  Each  criterion  is  then  described  in  terms 
of  constituent  elements  which  can  oe  either  directly  measured 
(parameters)  or  modeled  (submodels).  The  parameters  are  then  related 
to  the  criteria  via  the  submodels  and  the  resulting  criteria  are 
combined  with  their  relative  weights  in  the  form  of  the  criterion 
function  (CF) .  Following  additional  analysis,  an  optimal  candidate 
system  (selected  based  on  CF  values)  is  identified,  tested  further, 
and  implemented  based  on  plans  formulated  in  the  Detail  Activities. 
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Chapter  3  focused  on  the  issue  of  identifying  the  optimal 
candidate  system  by  reviewing  some  of  the  foremost  general  and  design- 
based  quantitative  decision  making  approaches  which  have  been  used  or 
suggested  within  the  context  of  system  design.  General  approaches 
discussed  included  linear  programming,  goal  programming,  decision 
analysis,  and  dimensionless  analysis.  The  two  design-based  approaches 
reviewed  were  the  Tricotyledon  Theory  and  the  criterion  function. 
Although  each  of  the  approaches  have  demonstrated  merits,  the 
criterion  function  was  selected  as  the  best  starting  point  to  achieve 
the  desired  characteristics  stated  in  Chapter  1. 

Chapter  4  began  with  a  more  detailed  description  of  the  current 
method  of  criterion  function  implementation.  The  description  focused 
on  the  use  of  probability  theory  in  the  evaluation  of  criteria,  the 
measurement  of  interactions,  and  the  inclusion  of  relative  weights. 
The  chapter  then  continued  with  an  examination  of  the  logical  basis 
for  the  form  of  the  criterion  function.  Based  on  this  discussion,  the 
current  methods  were  reviewed  and  sources  of  inconsistency  identified. 
Specific  revisions  were  proposed  which  resolve  the  inconsistencies  in 
accordance  with  probability  theory.  In  particular,  the  revisions  drew 
on  the  concept  of  criteria  intersections ,  which  may  be  affected  by  the 
presence  of  interaction.  Finally,  a  discussion  was  presented 
concerning  the  use  of  empirical  distributions  as  an  alternative  to 
continuous  approximations . 


277 


In  Chapter  5,  a  demonstration  of  the  revised  approach  was 
accomplished  using  the  design  of  auxiliary  power  units  for  NASA’s 
space  shuttle.  The  demonstration  was  presented  using  the  format  of 
the  framework  design  methodology.  Four  decision  criteria  (and  their 
intersections)  with  continuously  varying  relative  weights  were  used  to 
evaluate  a  set  of  4,753  identified  candidate  systems.  The  proposed 
criterion  function  method  was  implemented  using  a  series  of  BASIC 
programs  run  on  personal  computers .  The  optimal  candidate  system  was 
identified  and  generalizations  were  suggested  based  on  a  review  of  the 
fifty  top-rated  systems. 


Conclusions 

At  the  conclusion  of  Chapter  1,  the  research  objective  was 
defined  to  be  the  development  of  a  prescriptive  design  decision  making 
approach  which: 

1)  supports  comparative  evaluation  of  a  large  number  of  design 
alternatives , 

2)  considers  multiple,  potentially  conflicting  objectives, 

3)  considers  both  objective  and  subjective  relevant  factors, 

4)  permits  relative  weights  which  vary  both  among  criteria  and 
within  the  range  of  possible  criteria  values, 

5)  seeks  decisions  which,  within  the  context  of  the  defined 
problem,  are  optimal  in  the  sense  suggested  by  Keen  (1977)  and 
discussed  earlier  in  Chapter  1,  and 

6)  considers  interaction  among  criteria. 


As  a  result  of  the  discussion  in  Chapter  3,  the  criterion 
function  was  selected  as  the  most  promising  basis  for  the  achievement 
of  the  research  objective.  However,  it  was  acknowledged  that  the 
current  method  of  criterion  function  implementation  was  capable  of 
producing  results  which  are  inconsistent  with  its  basis  in  probability 
theory.  It  was  therefore  observed  that  a  key  goal  for  the  success  of 
the  research  would  be  the  "clarification  of  the  criterion  function 
implementation  technique  to  produce  figures  of  merit  on  the  range  0  to 
1.0,  consistent  with  the  underlying  theory." 

In  order  to  achieve  this  goal,  Chapter  4  identified  specific 
inconsistencies  in  previous  CF  implementation  methods,  identified  the 
sources  of  these  inconsistencies,  and  proposed  revisions  which 
resolved  them.  Chapter  5  then  demonstrated  the  revised  method  on  an 
actual  design  problem. 

During  the  accomplishment  of  these  activities ,  the  following 
conclusions  were  obtained  which  are  of  primary  importance  with  respect 
to  the  research  objective. 

1.  Criteria  intersections  were  meaningfully  distinguished  from 
criteria  interaction. 

2.  Computation  of  criteria  intersections  (Xjj)  required  the  specified 
revisions  to  achieve  consistency  with  probability  theory. 


3.  Computation  of  criteria  relative  weights  (A^,  Ajj  ,  etc.)  required 
the  specified  revisions  to  achieve  consistency  with  probability 
theory . 

4.  The  resulting  revised  approach  to  criterion  function 
implementation  satisfies  the  stated  research  goal. 

In  addition,  the  following  conclusions  of  secondary  importance 
emerged  from  the  activities  of  Chapters  4  and  5. 

5.  Additional  insight  was  gained  into  the  logical  basis  of  the 
criterion  function,  especially  with  respect  to  the  meaning  of  criteria 
intersection  terms  and  intersection  relative  weights. 

6.  Empirical  distribution  functions  were  demonstrated  as  a  useful 
alternative  to  continuous  approximations  in  the  evaluation  of 
candidate  system  performance. 

7.  The  problem  of  NASA's  space  shuttle  APU  design  provided  insight 
both  in  terms  of  the  applicability  of  a  theoretical  approach  to 
decision  support  and  in  terms  of  preliminary  results  based  on  data 
currently  available. 

8.  The  proposed  methods  were  demonstrated  using  only  simple  BASIC 
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programs  and  personal  computers ,  thereby  supporting  ease  of 
implementation  by  practitioners. 

Any  research  is  subject  to  limitations.  In  addition  to  the 
specific  assumptions  mentioned  at  the  outset  of  Chapter  5,  the 
following  limitations  (all  previously  identified)  are  acknowledged. 

1.  The  preliminary  and,  therefore,  tentative  nature  of  the  data  and 
estimates  used  for  the  demonstration  in  Chapter  5 . 

2.  The  depth  and  sophistication  of  the  criteria  modeling  activities. 

3.  The  lack  of  resources  necessary  to  accomplish  thorough  sensitivity 
analysis . 

4.  The  difficulty  of  empirically  validating  the  superiority  or 
optimality  of  the  proposed  method. 

The  first  three  limitations  pertain  to  the  demonstration  and  were 
addressed  in  Chapter  5.  With  respect  to  the  fourth  limitation 
(empirical  validation) ,  it  was  acknowledged  in  Chapter  1  that 
establishing  the  optimality  of  multi -criteria  decision  techniques  is 
inherently  problematic.  It  is  precisely  the  issue  of  defining  the 
"best"  solution  which  differentiates  alternative  approaches.  Direct 
comparison  of  the  solutions  produced  by  various  methods  is  therefore 
impaired  by  the  differences  in  the  assumptions  employed  in  each 
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approach.  It  was  for  this  reason  that  we  emphasized  consistency  with 
the  underlying  theory  as  the  most  important  guide.  However,  empirical 
illustrations  have  been  used  where  possible  to  compare  the  proposed 
revisions  with  previous  implementation  methods. 

Recommendations  for  Future  Research 

Given  the  breadth  of  issues  discussed  and  the  flexibility  of  the 
methods  presented  here,  numerous  opportunities  for  future  research  are 
apparent.  Those  described  in  the  following  paragraphs  would  seem  to 
represent  particularly  important  areas. 

First,  a  useful  aid  to  researchers  In  this  area  would  be 
empirical  studies  concerning  the  design  methodologies  currently  used 
in  practice  and  the  success  with  which  they  support  structured 
decision  making.  While  numerous  citations  were  presented  in  this 
research  concerning  various  methodologies  and  their  use,  virtually  all 
of  these  works  are  biased  In  the  sense  that  their  purpose  is  primarily 
to  describe  or  promote  a  particular  approach.  There  is  an  apparent 
lack  of  descriptive  field  work  (e.g.  surveys)  which  would  provide 
insight  concerning  unaddressed  needs  of  practitioners. 

A  second  research  need,  also  empirical  in  nature,  is  a  method  to 
support  empirical  validation  or  "fair"  comparison  of  multi-criteria 
decision  techniques.  Direct  comparison  of  the  results  produced  by 
alternative  methods  will  typically  reveal  simply  different  answers 


attributable  to  different  assumptions.  While  descriptive  techniques 
may  be  validated  using  direct  interaction  with  decision  makers,  a 
basis  is  needed  to  facilitate  empirical  comparison  of  prescriptive 
methods  without  sacrificing  the  treatment  of  subjective 
considerations . 

Third,  it  would  seem  appropriate  to  pursue  work  promoting  the 
value  of  the  criterion  function  based  on  the  merit  of  its  explicit 
consideration  of  intersection  relative  weights.  The  capability  to 
simultaneously  trade  off  both  marginal  criteria  performance  and 
various  combinations  of  balance  is  a  unique  strength  of  the  method 
which  would  appear  to  be  only  vaguely  approached  by  goal  programming. 

As  mentioned  in  Chapter  4,  the  normalization  scheme  employed  for 
relative  weights  is  a  very  effective,  yet  subtle  means  of  insuring 
consistency  in  the  evaluation  process.  Further  work  is  merited  to 
fully  describe  and  explore  its  significance  both  in  the  context  of  the 
criterion  function  and  in  other  decision  making  techniques. 

Another  suggestion  is  the  explicit  consideration  of  relative 
weight  tradeoffs  as  a  supplement  to  the  currently  proposed  sensitivity 
analysis.  It  is  quite  likely  that  in  some  applications,  CF  results 
may  be  highly  responsive  to  specific  changes  in  relative  weights, 
thereby  focusing  the  attention  of  decision  makers  on  the  most  relevant 
aspects  of  the  evaluation.  By  contrast,  other  implementation  results 
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may  prove  quite  insensitive  to  modifications  of  relative  weights, 
thereby  increasing  the  confidence  of  the  decision  makers. 

Another  issue  is  the  apparent  goal  of  CF  (as  described  in  Chapter 
4)  to  seek  that  candidate  system  which  best  avoids  a  bad  choice  while 
many  other  multi-criteria  approaches  pursue  proximity  to  the  theoretic 
ideal.  (These  two  alternative  approaches  were  distinguished  in 
Chapter  4  by  the  nature  of  their  isopreference  lines.)  What  are  the 
tradeoffs  between  these  two  general  approaches  and  do  cases  exist 
where  one  or  the  other  is  particularly  appropriate? 

A  closely  related  issue  is  raised  by  the  possible  addition  of  a 
third  dimension  to  the  current  matrix  of  eight  CF  models.  The  third 
dimension  could  be  defined  in  terms  of  (at  least)  three  discrete 
cases:  Ajj  are  positive,  negative,  or  zero.  Again,  isopreference 
curves  may  serve  as  a  useful  tool  in  comparing  the  appropriateness  and 
effects  of  these  three  cases . 

Finally,  an  obvious  opportunity  might  be  to  pursue  the  use  of  the 
design  morphology  and  the  criterion  function  approach  for  NASA's  APU 
and  related  design  decisions  (e.g.  EMA) .  While  the  parameters, 
submodels,  and  even  candidates  considered  would  be  expected  to  change 
in  a  more  formal  application,  it  was  apparent  that  the  criterion 
function  responded  directly  to  important  aspects  of  the  decision 
problem  not  normally  accommodated  by  other  approaches. 


Appendix  A 


THEORETICAL  BASIS  FOR  THE  USE  OF  PROBABILITY 
IN  CRITERIA  MEASUREMENT 


Note:  The  material  in  this  appendix  is  essentially  quoted  directly 
from  Ostrofsky  (1977d:  332-345).  It  has  been  altered  only  to  the 
small  degree  necessary  to  ensure  grammatical  and  formatting 
appropriateness  in  its  present  context.  The  discussion  begins  with 
consideration  of  a  single  criterion,  proceeds  to  the  extension  to  two 
criteria,  and  concludes  with  the  extension  to  multiple  criteria  in 
general . 


Embedding  a  Criterion  in  a  Probability  Space 


Define 

7  -  elementary  event  resulting  from  candidate  system, 
ft  -  set  of  all  elementary  events  7;  each  event  in  ft  implies 
the  occurrence  of  x, 

x  -  design  criterion  with  the  range  x^n^xs^^ , 

T  -  Boolean  field,  i.e.,  a  class  of  sets,  C,  such  that  if  events 
A^  and  A2  «  C,  then  A^c,  A^  U  A2 ,  Aj  A  A2  are  also  in  C. 
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Define  a  property  of  event  A  such  that  the  event  that  A  does  not 
hold  is  the  set  of  all  elements  in  ft  not  belonging  to  A,  or,  the 
complement  of  A,  represented  by  Ac.  It  follows  that  the  complement  of 
ft  is  the  null  set  $. 

From  this  notation  an  elementary  event  can  be 

X(7i)  -  y-i  Eq.  A.  1 
or 

X'^xj)  -  Eq.  A. 2 

Equation  A.l  simply  states  that  each  occurrence  of  the  random  variable 
implies  a  particular  x^  and  similarly  Equation  A. 2  states  that  each 
occurrence  of  an  x^  implies  a  particular  y^;  this  notation  can  be  read 
as  "the  preimage  of  x^  is  y^."  Hence,  the  elementary  event  y^  occurs 
if  an  only  if  the  respective  value  of  the  criterion,  x^,  occurs  since 
7£  has  been  defined  on  the  space  of  x^  such  that  a  unique  x^  exists 
for  each  y^. 

Further,  y^  can  be  defined  as  the  occurrence  of  a  given  set  (or 
subset)  of  x^  such  that  the  random  event  X(y^)  implies  {x^}.  Hence 
the  random  event  can  be  the  occurrence  of  a  design  concept  instead  of 
the  candidate  system. 


Some  additional  comments  on  x  are  appropriate  at  this  point.  In 
the  context  of  the  design  morphology,  x  is  considered  the  resulting 
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performance  of  a  candidate  system  for  a  given  concept.  Since  x  is 
discrete,  it  should  be  the  optimal  value  of  the  criterion  which  can  be 
obtained  from  that  particular  candidate  system.  This  is  an  important 
limitation  on  the  definition  of  "candidate  system"  since  criterion 
performance  other  than  optimal  can  be  obtained,  and  these  alternatives 
would  then  define  additional  candidates  for  a  given  concept.  The 
importance  of  this  limitation  becomes  more  evident  when  multiple 
criteria  are  considered  and  leads  to  the  important  property  of  being 
able  to  identify  the  performance  of  the  total  population  of  candidates 
in  Q. 


For  each  A  c  T,  a  value  P(A)  can  be  assigned  and  is  considered 
the  probability  of  A.  P(A)  is  then  a  set  function  over  the  members  of 
T.  The  following  axioms  are  offered  for  the  function  P  by  Rao  (1965): 


Axiom  1.  P(A)2:0,  A  e  T. 


Axiom  2.  Let  A^,  . .  .  ,A^  (A  e  T)  be  disjoint  sets  whose  union  is  0. 
Thus  any  elementary  event,  A^,  has  one  and  only  one  of  k  possible 
descriptions,  A^,...,A^.  Then  the  relative  frequencies  of  the  events 
A^ , . . . , A^  must  sum  to  1.  Thus 

®  k 

UAi  -fl,  Atft  A<  -4  for  all  i/j  where  SPCA^  -  1  Eq.  A.  3 
i-1  i-1 
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Then  a  set  function  P  defined  for  all  sets  of  T  and  satisfying  axioms 
1  and  2  is  called  a  probability  measure.  The  proof  of  Equation  A. 3  is 
given  by  Rao  (1965) .  It  then  becomes  apparent  that 


P(UAi)  -  SPCAi)  Eq.  A. 4 

for  any  countable  union  of  disjoint  sets  in  T  whose  union  also  belongs 
to  T .  Further , 

(UAi)U(UAi)c  -  0  -  (UAi)cUA1UA2  ...  Eq.  A. 5 

and  hence 

P(UAi)+P[(UAi)c]  -  1  -  P[(UAi)c]+P(A1)+P(A2)+  ...  Eq.  A. 6 


Since  a  Boolean  field  need  not  contain  all  countable  sequences  of 
sets,  the  field  which  does  contain  all  such  unions  is  the  Borel  Field, 
P,  sometimes  called  a  field  (Loeve,  1963;  Rao,  1965).  Given  a  field  T 
(or  any  collection  of  sets)  there  exists  a  minimal  Borel  field,  P(T) 
which  contains  T.  Since  arbitrary  intersections  of  Borel  fields  are 
also  Borel  fields,  the  intersection  of  all  Borel  fields  containing  T 
is  precisely  the  minimal  Borel  field,  P(T) .  Since  a  Borel  field  is 
also  a  Boolean  field,  Axioms  1  and  2  and  Equations  A. 3 -A. 6  apply. 
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Then  for  P(T)  there  exists  a  unique  function  P*  which  is  the 
probability  measure  on  P(T)  for  the  set  A.  Rao  (1965)  defines  P*  as 
follows:  "Consider  a  set  A  in  y9(T)  and  a  collection  of  sets  A^  in  r 
such  that 


Then 


A  is  a  subset  of 


CO 


UAt 

i-1 


P*(A)  -  inf  AjSPCAi) . " 


The  calculus  of  probability  is  based  on  the  space  fl  of  elementary 
events  7;  a  Borel  field,  p,  of  sets  in  0;  and  a  probability  measure  P 
on  p.  This  triplet  (£1,  p,  P)  is  called  a  probability  space.  The 
calculus  is  as  follows: 


cx  -  Alyjl i-1, . . . ,k) 

Eq. 

A. 7 

and 

Cx  “  Uili-1, . . .  ,k) 

Eq. 

A. 8 

and  from  Equations  A.l  and  A. 2 

X(7i)  -  X(Ai)  -  Xi 

Eq. 

A. 9 

and 


X'^(x^)  is  a  subset  of  f)  and  X(A^)  is  a  subset  of  p  Eq.  A.  10 
Then 

P(Ai)  -  PlX^iJ-Xi]  Eq.  A. 11 

If  x  is  a  design  criterion  with  known  range  x^-fp^x^x^y,  a  random 
variable  X(y)  can  be  defined  such  that  X(y)  -  x  for  a  given  candidate 
system.  X(y)  then  is  defined  to  be  a  criterion  random  variable  whose 
probability  density  results  from  the  set  of  candidate  systems.  Then 
the  set  of  candidate  systems  which  have  been  synthesized  for  the 
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design  during  the  feasibility  study  implies  a  set  of  values  x  from 
which  inferences  can  be  made  concerning  the  possible  set  of  values  Cx 
for  the  design,  and  the  sample  space,  0,  is  the  set  of  possible 
candidate  systems  for  the  design,  and  inferences  concerning  the 
various  attributes  of  0  can  be  made  from  Cx  implied  by  the  candidates 
which  have  been  synthesized. 

Then  T  is  the  class  of  all  sets  of  x  which  result  from  the  set  of 
recognized  candidate  systems.  If  the  set  of  recognized  candidate 
systems  is  exhaustive  for  the  admissable  range  of  x,  then  the  union  of 
all  sets  of  x  is  a  Borel  field,  fi.  When  the  union  of  all  candidate 
systems  within  the  admissable  range  of  x  is  precisely  the  intersection 
of  all  Borel  fields  containing  T,  the  minimal  Borel  field,  £(D  is 
achieved,  that  is,  the  smallest  class  of  candidate  systems  which 
contain  all  the  admissable  x. 

When  considering  a  single  design  criterion  the  elementary  event  7 
is  identically  equal  to  the  design  criterion  x.  Further,  the  event  A 
for  the  single  criterion  is  equivalent  to  the  event  7^  (see  Equations 
A.  9  and  A.  10).  Then,  for  each  x  e  fi  a  measure  P(x)  can  be  assigned. 
Thus  a  set  function  P  over  the  members  of  0  is  achieved  and  can  be 
considered  a  probability  set  for  x.  Hence  a  design  criterion  x  can  be 
transformed  into  a  probability  space,  (0,  f),  P) ,  such  that  P[X(7>-x] 
is  analogous  to  P(A) .  Then  Rao's  (1965)  Axioms  1  and  2  apply: 


1.  03P[X(7)-x]<;1. 
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00  CO 

2.  Ux£-0,  for  all  i/j  where  SP[X(7£)-X£]-1. 
i-1  J  i-1 

Consequently  the  theorems  and  axioms  of  probability  apply  to 
P[X(7^)-X£]  as  well  as  the  notions  of  probability  densities  and 
probability  distributions. 

Fitting  the  derived  criterion  distribution  function  to  one  of  the 
standard  distribution  forms  facilitates  computation.  Thus  a  brief 
summary  of  distribution  properties  is  presented  at  this  point  in  the 
discussion.  These  properties  are  well  known,  and  their  proofs  are 
presented  in  most  texts  on  probability  theory.  This  discussion 
follows  the  presentation  and  proofs  given  by  Rao  (1965)  and  lists  the 
more  relevant  properties. 


1.  The  distribution  function  (d.f.)  defined  on  R, 

the  real  line,  is 

F(x) 

-  P{7:  X(7)<x)  -  P[X-1(  —  ,x)] 

Eq.  A. 12 

Since  xaac,,^. 

F(x)  -  PtX'^Xnin.x)] 

Eq.  A. 13 

2. 

1 

8 

1 

o 

Eq.  A. 14 

F(»)  -  F(xmax)  -  1 

Eq.  A. 15 

3.  F(x)  is  nondecreasing: 

P[X_1(  -«,x2)  ]  *  PlX^C-o.Xi)] 


Eq.  A. 16 
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4.  F(x)  is  continuous  at  least  from  the  left. 

5.  The  criterion  random  variable  on  (0,  f),  P)  induces  a  probability 
measure  on  (R,  f}\) ,  and  this  probability  measure  is  completely 
specified  by  F(x) . 

6.  The  criterion  random  variable  X(y)  defines  a  Borel  field  of  sets 
which  are  subsets  of  f);  /3X  is  a  sub -Borel  field. 

7.  A  function  f  which  maps  points  of  Rn,  the  n-dimensional  space  into 
R  is  said  to  be  a  Borel  function  if 

Scf}  where  f^-(S)  e  Eq.  A.  17 

where  S  is  a  set!  of  points. 

The  Class  of  Subsets  for  Two  Criteria 

Let  R^  be  defined  as  the  set  of  all  elementary  events  with 
elements  7;  each  event  in  R^  implies  the  occurrence  of  (x^,X2). 

xi  -  design  criterion  1  with  range  x^  ^^SXjSIxi  max. 

X2  -  design  criterion  2  with  range  X2  min^x2:Sx2  max- 
A  -  an  event  which  denotes  the  occurrence  of  any  combination  of 
elementary  events  (i.e.  any  combination  of  the  joint 
occurrences  of  x^  and  X2i). 

T  -  Boolean  field,  i.e.  a  class  of  sets,  C,  such  that  if  events 
A^  and  A2  f  C,  then  A^c,  A^UA2»  A^AA2  are  also  in  C. 
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Then  the  criterion  random  variable  is  defined, 

X(7i)  “  X(7ii,72i>  "  tX<ni>.  X(72i>]  Eq.  A. 18 

where 

X  -  (X1,X2)  -  (x1,x2)  Eq.  A. 19 

so  that  usually 

X‘1(x1,x2)  -  7t 

Equation  A.  18  states  that  the  elementary  event  7^  is  the  joint 
occurrence  of  a  value  for  each  of  the  two  subevents,  7^  and  721 .  and 
that  the  criterion  random  variable,  X,  is  the  joint  occurrence  of  a 
value  from  each  of  the  two  design  criteria.  Then  from  Equation  A.  19 
each  occurrence  of  (xllfx2i)  implies  a  particular  7^  or  the  preimage 
°f  (7ii.72i)  is  7i-  Then,  in  the  context  of  design,  (x].i.x2j.) 
represents  the  joint  occurrence  of  a  real  number  for  each  criterion, 
respectively,  which  results  from  a  candidate  system.  The  set  of 
candidatr  systems  which  is  exhaustive  then  provides  the  ability  to 
define  R^. 

It  is  recalled  that  each  candidate  system  provides  one  number  for 
(xii,X2i)  so  that  variations  in  the  performance  of  each  xj^  are 
considered  to  yield  different  candidates.  This  restriction  enhances 
the  ability  to  discriminate  among  candidates  which  yield  superior 
numbers  on  only  one  of  the  x's. 


293 


The  Probability  Set  Function  and  Borel  Field 

Two  single -valued  real  functions  which  map  ti  into  R^,  two- 
dimensional  space,  result  in  a  two-dimensional  criterion  random 
variable.  Parzen  (1960)  describes  the  probability  function,  P [ . ] ,  as 
"a  numerical  2- tuple -valued  random  phenomenon  whose  value,  P(A)  at  any 
set  A  of  2 -tuples  of  real  numbers  represents  the  probability  that  an 
observed  occurrence  of  the  random  phenomenon  will  have  a  description 
lying  in  the  set  A."  Parzen  further  identifies  the  probability,  P[ . ] , 
as  analogous  to  a  distribution  of  unit  mass  of  some  substance  called 
probability  over  a  two-dimensional  plane  which  is  calibrated  with  a 
coordinate  system.  Then  for  any  set  A  of  2-tuples  P(A)  states  the 
weight  of  the  probability  substance  distributed  over  the  set  A. 

Then  Axioms  1  and  2  originally  defined  for  one  single-valued  real 
function  apply  equally  well  to  the  two  single-valued  real  functions 
when  A  and  T  are  suitably  redefined.  Hence  Equations  A.4-A.7  apply 
for  the  two  single-valued  functions. 

The  Borel  field,  /32 ,  is  then  defined,  as  in  the  single  variate 
case,  as  the  field  which  contains  all  countable  sequences  of  sets  and 
their  unions.  Again,  the  minimal  Borel  field,  £2(r),  is  defined  from 
the  intersection  of  all  Borel  fields  containing  precisely  T. 

Having  defined  the  sample  space  in  terms  of  R2 ,  the  Borel  field 
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as  fP ,  and  the  probability  measure  as  P,  the  criterion  probability 
space  becomes  (R2,  fP ,  P) . 


Further , 

A^-  -  (71).  a  subset  of  0^ 
A2  -  (72),  a  subset  of  02 


Eq.  A. 20 


and 

A  —  A^*A2,  a  subset  of  R2  -  0^*02  Eq.  A. 21 

Also,  the  preimage  of  the  criterion  random  variable  (X^^)  is 
contained  in  the  Borel  field;  hence  the  preimage  of  each  candidate 
system  is  also  in  the  Borel  field: 


X'1(xii,X2i)  is  a  subset  of  fP  Eq.  A.  22 

Hence 

PCAi)  -  P[X(7i)  -  <Xli,x2i>J  Eq.  A. 23 


Joint  Probability  Distribution  and  Conditional  Dis.trib.u£igp 

Let  (Xi,X2>  be  the  two-criterion  random  variable  having  the 
probability  space  (R2,  (P  ,  P) .  Then  Wilks  (1962)  defines  E  to  be  the 
interval  ( -«o,  -«o;X^  ,X2]  in  R2  and 

F(xi,X2>  -  P(E)  -  P[7 1X1(7) ^xii >X2 (7)^X2 Eq.  A. 24a 
which  is  clearly  a  single -valued,  real,  non-negative  function  of 
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(X^,X2)  in  R2.  Then  any  interval  I  of  the  form  (xn,X2i;x^j  ,X2j )  £  02 
since 

1  "  tE(xlj,x2j)-E(xli,x2j)-E(xlj,x2i)-E(xli,x2i)l  Ecl-  A-24b 

where  i  and  j  are  two  candidate  systems .  Then  the  probability  that 
(X1,X2)  £  I  is 

P[(X1,X2)£l]  -  F(xli,x2i)-F(xli,x2j)-F(x1j ,x2i)-F(x1j ,x2j) 

Eq.  A. 25 

Wilks  calls  the  expression  on  the  right  of  this  equation  the  "second 
difference  of  F(x^,X2>  over  I"  and  denotes  this  difference  by 

5^2F(x^,X2)  Eq.  A. 26 

Figure  55  relates  to  the  various  quantities  as  follows: 

F(x^j,X2j)  -  doubly  shaded  area. 

F(xn,X2j)  -  area  shaded  vertically. 

F(x^j ,X2i>  -  area  shaded  horizontally. 

F(x^£,X2^)  “  area  below  and  to  the  left  of  coordinates. 
5^2F(xi,X2)  -  unshaded  area  below  and  to  the  left  of 
(Xlj,X2j) 
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Further , 

F(-«o,X2)  -  0  Eq.  A. 27 

F(x^,-«)  -  0  Eq.  A. 28 

F(«,«o)  -  1  Eq.  A. 29 

Also,  if  (X^,X2>  is  a  criterion  random  variable  with  probability  space 
(R^,  f)?- ,  P) ,  a  unique  function  F(x^,X2>  exists  defined  by  Equation 
A. 24a  at  every  point  in  R^.  Conversely,  a  function  F(x^,X2>  defined 
by  Equation  A. 24  at  least  continuous  from  the  left  and  conforming  with 
Equations  A.  26 -A.  29  uniquely  determines  a  probability  space  (R^,  p% , 
P) .  Further  discussion  of  these  two  statements  is  given  by  Wilks 
(1962).  Hence,  F(x^,X2)  is  called  the  distribution  function  of  the 
two-dimensional  random  variable  (X^,X2>.  F(x^,X2)  may  also  be 

referred  to  as  a  "bivariate  criterion  distribution  function." 

The  marginal  distributions  are 

Fi(xii)  -  FCxii,®)  -  P[Xi(7)<XijJ  Eq.  A. 30 

F2(x2i)  -  F(»,x2i)  -  P[X2(7)^X2i]  E<*-  A-31 

If  one  considers  the  total  probability  function  as  being 
distributed  in  the  (X^,X2>  plane  in  accordance  with  F(xi,X2>,  then 
Fi(xi>  is  the  projection  on  the  X]i  axis  and  F2(x2>  is  the  projection 
on  the  X2  axis.  Hence  it  follows  that  F^(x^)  and  F2<X2)  determine  the 
probability  spaces 

[R1<1),/9ia),P<D]  and  [R2d>  ,02(1)  ,P(2)  ] 
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respectively,  where  and  R2^^  are  the  sample  spaces  denoted  by  0 

in  the  one - dimens ional  case. 

When  (X^,X2)  is  a  bivariate,  criterion  random  variable,  having 
F(x^,X2),  Wilks  (1962:  43)  shows  that  a  necessary  and  sufficient 

condition  for  X^  and  X2  to  be  independent  is  that 

F(x1,x2)  -  F(x1)*F(x2)  Eq  A. 32 

Further,  let  I  be  the  set  in  for  which  x^^<x^<x^j  such  that 
P(I)  >  0;  then 


PC-  w 

F(xi,X2|I)  - .  Eq.  A. 33 

P(I) 

where  E  is  defined  as  in  Equation  A.  24a  to  be  (-<*>,  -«,X^  ,X2 ]  .  Hence 
the  conditional  probability  of  a  bivariate,  criterion  random  variable 
given  any  set  in  the  sample  space  R^  is  the  expression  of  Equation 
A. 33. 


When  F^(xi)  is  the  marginal  distribution  function  of  x^, 


F(xij ,x2)-F(xli,x2) 

F(xii,X2H)  - . 

Fl(xij)-Fl(xU) 


Eq.  A. 34 
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When  the  limit  on  the  right  side  exists  as  xj^  approaches  x^j  then  it 
is  denoted  by 


F(X2|X]4)  -  limit  F(x^4,X2|I)  Eq.  A. 35 
xu->xij 

If  F(x2|x^j)  exists  for  every  value  of  X2,  then  the  F(x2|x^j)  is 
a  distribution  function  and  has  all  the  properties  of  a  d.f.  and  is 
called  the  conditional  distribution  function  of  X2  given  X^-x^j ,  or, 
when  there  is  no  possibility  of  ambiguity,  F(x2|xj^),  where  X^  is 
regarded  as  a  parameter  of  given  value,  not  as  a  random  variable. 
Wilks  (1962)  identifies  X2 |x^  as  a  one -dimensional  random  variable 
whose  distribution  function  F(x2|x^)  for  a  fixed  value  of  X^  is 
defined  at  all  points  in  the  X^X2  plane  for  which  X^  -  x^j  in  such  a 
way  that  F(x2j|x]j)  is  approximately  the  amount  of  probability  (or 
probability  density)  lying  along  the  portion  of  the  line  X^  -  x^j  for 
which  X2^X2j • 


Finally,  for  every  set  B  of  2 -tuples  (Parzen,  1960) 


P(B) 


-fl 


f (x^,X2)dx^dx2 


B 


Eq.  A. 36 


where  f(x^,X2)  is  the  criterion  probability  density  function.  Then 
the  criterion  distribution  function  becomes 
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Xfc  -  kth  design  criterion  with  the  range  min:SX:Sxk  max> 
k-1, . . . ,n. 

A  -  an  event  which  denotes  the  occurrence  of  any  of  elementary 
events . 

T  -  Boolean  field  defined  in  a  manner  similar  to  that  used 
previously. 


If  the  real  line  (-«,<*>)  is  represented  by  R  and  the  n-dimensional  real 
Euclidean  space  by  Rn,  then  the  points  of  Rn  can  be  represented  by  the 
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set  of  real  coordinates  (x^ . xn) ,  where  x^  is  defined  as  shown 

above.  Then  the  criterion  random  variable  is  defined  for  the  ith 
candidate  system  as 

X(7i)  -  (xlif . . . ,xki)  Eq.  A. 40 

and 

X_1(xii, . . . ,xki)  -  Eq.  A. 41 


Thus  for  the  design  of  a  system  with  multiple  criteria,  each  candidate 
system  provides  one  real  number  for  each  x^,  k-l,...,n.  As  before, 
any  variation  in  the  system  performance  which  leads  to  a  different 
number ,  x^i '  to  be  considered  as  the  design  value,  implies  that  the 
system  must  be  considered  as  a  different  candidate.  Consequently,  as 
in  the  bivariate  case,  an  enhanced  ability  to  discriminate  among 
candidates  exists  as  well  as  the  ability  to  define  an  increased  number 
of  candidates  from  which  to  choose. 


The  Probability  Set  Function  and  Borel  Field 

When  n  single-valued,  real  functions  exist  which  map  ft  in  Rn  the 
collection  may  be  called  an  n-dimensional  criterion  random  variable. 
As  in  the  bivariate  case  Rao's  (1965)  Axioms  1  and  2  apply  to  the  n- 
variate  probability,  and  Equations  A. 4 -A. 7  apply. 


Also,  the  Borel  field,  ^n,  is  again  defined  as  the  field  which 


302 


contains  all  countable  sequences  of  sets  and  their  unions.  Further, 
the  minimal  Borel  field,  /5n(r) ,  is  defined  from  the  intersection  of 
all  Borel  fields  containing  precisely  T. 


Again,  the  probability  space  becomes  (Rn,  0n,  P) ,  and  Equations 
A.7-A.10  apply  with  the  newly  defined  A^ ,  7^,  and  /Jn,  and  Equation 
A. 11  becomes 


P(Ai)  -  P[X(7i)  “  <xn,...,xki)]  Eq.  A. 42 


The  previous  results  extend  directly  to  the  n-dimensional  case. 

If  (Xx . Xjj)  is  an  n  criterion  random  variable  whose  probability 

space  is  (Rn,  £n,  P)  and  E(xn ,  .  .  .  ,xk^)  is  the  set  (-«*> . - 

• ;  Xj_ ,  .  .  .  ,  X^)  in  Rn ,  then 


F(x1,...,xn)  -  P(E)  -  P[7:Xk(7)<xki,k-l, . . . ,n]  Eq.  A. 43 


where  i-ith  candidate  system.  Thus,  as  in  the  two  criterion  case, 

F(x^ . xk)  is  clearly  a  single -valued,  real,  and  non-negative 

function  of  (X^,...,Xk)  in  Rn. 

Any  interval  I  e  Rn  of  the  form  (x^ . ;  xlj>--->xnj) 

belongs  to  Rn  since 
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xki)l 
Eq .  A . 44 

Then  the  probability  that  (X^ . Xn)  e  I  can  be  found  in  terms  of 

F(X! . Xn)  by 

P[(X! . Xn)  £  I]  -  F(xXj . Xn j  ) 

-  [F(x]4,x2j .... ,Xnj)  +  .  .  . 

+  *(xlj . %-i.j.Xni)] 

+  >x2i  »x3j . Xnj)  +  ••• 

+  F(x^j , . . . ,Xn.2,j •xn-l,i>xni^ 


+  (-l)nF(xli . Xni)  Eq.  A. 45 

Note:  This  notation  is  a  modified  form  of  that  used  by  Wilks  (1962); 

also  see  Parzen  (1960) .  Then  the  right  side  of  this  equation  can  be 
denoted  by  6 jnF(x^ , . . . ,Xn)  over  I  so  that 

P[(X1(  .  .  .  ,Xn)  e  I]  -  «!kF(Xl . xk)2:0  Eq.  A. 46 


By  argument  similar  to  Equation  A. 27 


F(-«o,X2,  .  .  .  ,Xn)  -  ...  -  F(XltX2 . Xn_1(-®)  -  0  Eq.  A. 47 


and 


J(xl1 . xnj ) 

•  tE(xlj  ,x2j . xnj)u-  •  -UE(xlj  ,x2j  ....  ,xn-l, 


F(® . ®)  -  1 


Eq.  A. 48 
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Further , 

F(xlf . .  .  .xi-i,Xi+0.xi+ll . .  •  .Xn)  -  F(xj_ . x„)  i-1 . n 

Eq.  A. 49 

which  states  that  the  function  is  continuous  on  the  right.  Further, 
F(x^,  .  .  .  .xjj)  is  unchanged  if  any  set  of  x^,  say  (x^£ ,  .  .  .  .x^)  ,  is 
replaced  by  <xli+0,  .  . .  .x^+q)  ,  respectively. 

Hence  if  is  an  n- dimensional  criterion  random  vector 

with  the  probability  space  (Rn,  (i n,  P) ,  there  exists  a  function 
F(x^,...,xn)  defined  at  every  point  in  Rn  by  Equation  A. 43  which  has 
the  properties  identified  in  Equations  A. 45 -A. 49. 


Conversely,  if  F(xi ,  .  .  .  .x^  is  defined  by  Equation  A. 43  and  has 
the  properties  of  Equations  A.45-A.49,  the  probability  space  (Rn,  pn , 
P)  is  obtained. 

The  function  F(x^ . xn)  can  be  called  the  multicriterion 

distribution  function  and  is  directly  analogous  to  the  multivariate, 
cumulative  distribution  function  of  probability  theory. 

Marginal  and  Conditional  Distributions 

The  marginal  criterion  distribution  function  of  X^,  F^(x^) ,  is 


defined  as 
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F^Xi)  -  F(xlt® . ®)  Eq.  A. 50 

with  the  other  marginal  distributions  being  similarly  defined: 

F i (Xi )  -  F(«, . . . .x^ . ®)  Eq.  A. 51 

More  generally,  the  marginal  distribution  function  of  (X^ ,  .  .  .  ,Xjc) , 
k<n,  is  defined  by 

Fl,...,k(xl . xk)  ”  F<xl . xk>“ . •)  Eq-  A. 52 

As  in  the  two-dimensional  case,  suppose  the  random  vector  X  to  be 
distributed  as  F(x1(  .  .  .  .Xjj)  ;  then  Fj^x^)  is  the  projection  on  the  x^ 
axis,  and,  more  generally,  F^Cx^)  is  the  projection  on  the  ith  axis  of 
the  n- dimensional  hyperplane,  each  criterion  having  a  one -dimensional 
probability  space. 

Wilks  (1962)  identifies  the  necessary  and  sufficient  condition 
for  (X^,...,Xic)  and  , .  . .  .Xjj)  to  be  independent  as 

F(x^ ,  .  .  .  ,x^)  “  F(x^ ,  .  .  .  , xk) *F ,  .  .  .  , x^)  Eq .  A .  53 

where  the  two  functions  on  the  right  are  marginal  distribution 
functions  of  the  respective  multicriterion  random  variable. 


Also,  are  mutually  independent  if  and  only  if 


-  F^(x1)*.  .  .*Fn(Xn^  E(l-  A-54 


Equation  A. 36  for  n  dimensions  becomes 


P(B) 


f (x^ , . . ■ , x^) dx^ , • • • , dx^ 


Eq.  A. 55 


and  Equation  A. 37  becomes 


F(xx . Xn) 


*ni  xli 

-]  ■  ]  £<X1’ 


-00  -00 


.  ,xT1)dx1 . dXj! 


Eq.  A. 56 


Also, 


6n 

f(xl . Xn)  - . F(xx . Xj,)  Eq.  A. 57 

6x^ , . . . ,  fXjj 

Conditional  distributions  for  the  multicriterion  case  are 
extensions  of  the  two-dimensional  case.  Let  X  -  (X^,...,Xn),  the  n- 
dimensional  criterion  random  vector  with  distribution  function 
F(x]_,  .  .  .  ,Xjj)  ,  and  let  F^  ^(x^, .  . .  .x^)  be  the  marginal  distribution 

function  of  (X^ . X^) ,  k<n,  and  let  I  be  the  set  in  Rn  for  which 

Xmj^XjjSXjjj ,  m-l,...,k;  then  the  projection  of  this  set  on  Rk,  the 
sample  space  of  (Xj. ,  . .  .  .X^)  ,  is  the  k-dimensional  interval 


I 


(xli , • • • , xki >  xlj i • • • >  xkj 1 ’ 


a  subset  of  R 


k 


Eq.  A. 58 
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Let  6jkFi  .k(xl>  •  •  •  >xk)  the  kth  difference  of 

k(xl . xk^  over  I  as  defined  in  Equations  A. 45  and  A. 46.  This 

kth  difference  is  P(I)  which  is  greater  than  0.  Now 


F(xlt  .  . .  .x^I) 


ptE(xl, . . . ,xn)  X1 
P(I) 


Eq.  A. 59 


and 

«IkF(xi . x^ 

F(xlj . xkj  •  xk+l . xj1)  “  V .  Eq.  a-60 

5I  Fl...k<xl . xk> 


where  the  numerator  is  the  kth  difference  of  F(x^,...,xn)  with  respect 
to  (x^ . xk)  for  fixed  values  of  (x^+^ , . . .  ,xTl)  . 


If  the  limit  on  the  right-hand  side  of  Equation  A. 59  exists  as 
xn  approaches  xjj . as  xk^  approaches  xkj  ,  that  is, 


limit  { F(xi j  i  •  •  •  . xkj  , xk+l ,  •  •  •  , x„j 1 1 )  ,  m-1 . k) 

xmi*>xmj 

“  F<xk+1- •  •  •  .^^lj . Eq.  A. 61 


then  this  is  called  the  conditional  random  vector 

(Xk+1 . Xn|X1,...,Xk) 

and  has  all  the  properties  of  an  (n-k) -dimensional  random  vector. 
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When  (XX . Xn) 

probability  density  is 

f (xk+l»  *  *  *  *  xn ( X1 » 

Also , 

f(x1( . . . ,xk) 


is  a  continuous  random  vector,  the  conditional 


f (X! , • « • , 

.  .  .  ,xk)  - .  Eq.  A.  62 

^1. . .k^xl> • * • »xk) 


-  f(xk|x1,...,xk.1) 

*  fl...k-l<xk-llxl . xk-2> 

* 

*  fi(x!>  Eq.  A. 63 


assuming  that  the  densities  of  all  the  conditional  random  variables  exist. 


Appendix  B 


SOURCE  LISTING  FOR  LITTLE-X.BAS 


LITTLE -X. BAS  COMPUTES  RAW  CRITERION  SCORES  FOR 
CANDIDATE  SYSTEMS  BASED  ON  PARAMETER  VALUES  STORED 
IN  SIX  DATA  FILES.  THE  RESULTS  ARE  STORED  IN 
DATA  FILES  B :  LITTLEX1 .  DAT  AND  B : LITTLEX2 . DAT . 


10  REM 
20  REM 
30  REM 
40  REM 
50  REM 
60  REM 
70  REM 
80  REM 
90  REM 
100  REM 

110  REM  THIS  MODULE  PERFORMS  INITIAL  HOUSEKEEPING 

120  REM  ********************************************************* 

130  PRINT  TIME$ 

140  OPTION  BASE  1 
150  DEFINT  F, I , J ,K 
160  MAX-4753 
170  DIM  Y(41) 

180  DIM  CAND(MAX) 

190  DIM  XlA(MAX) 

200  DIM  X2A(MAX) 

210  DIM  X3A(MAX) 

220  DIM  X4A(MAX) 

230  DIM  XMIN(4) 

240  DIM  XMAX(4) 

250  FOR  1-1  TO  4 
260  XMIN-9999 

270  XMAX-0 
280  NEXT  I 

300  REM  THIS  MODULE  READS  IN  PARAMATER  VALUES  FOR  ALL  CANDIDATE 
SYSTEMS 

310  REM  AND  INVOKES  THE  MODULE  WHICH  COMPUTES  RAW  CRITERION 
SCORES 


320  REM  ********************************************************* 

330  FIRST-1 
340  LAST-793 
350  FOR  F— 1  TO  6 

360  IF  F-l  THEN  INPUT  "PUT  1ST  INPUT  DISK  IN  B:  AND  PRESS 
ENTER" ;NIL$ 

370  IF  F-l  THEN  OPEN  "B : PARA- 1. DAT"  FOR  INPUT  AS  #1 
380  IF  F-2  THEN  OPEN  "B: PARA- 2 .DAT"  FOR  INPUT  AS  #1 
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390  IF  F-3  THEN  INPUT  "PUT  2ND  INPUT  DISK  IN  B:  AND  PRESS 

ENTER" ;NIL$ 

400  IF  F-3  THEN  OPEN  "B: PARA- 3 .DAT"  FOR  INPUT  AS  #1 

410  IF  F-4  THEN  OPEN  "B : PARA-4 . DAT"  FOR  INPUT  AS  #1 

420  IF  F-5  THEN  INPUT  "PUT  3RD  INPUT  DISK  IN  B:  AND  PRESS 

ENTER" ;NIL$ 

430  IF  F-5  THEN  OPEN  "B: PARA- 5 .DAT"  FOR  INPUT  AS  #1 

440  IF  F-6  THEN  OPEN  "B: PARA- 6 .DAT"  FOR  INPUT  AS  #1 

450  GOTO  750 

460  CLOSE  #1 

470  FIRST— LAST+1 

480  LAST— FIRST+7  9 1 

490  PRINT  F;"  ";TIME$ 

500  NEXT  F 

510  REM  ********************************************************* 

520  REM  THIS  MODULE  OUTPUTS  RAW  CRITERION  SCORES  TO  DATA  FILES 
530  REM  ********************************************************* 

540  INPUT  "PUT  1ST  OUTPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

550  OPEN  " BrLITTLEXl.DAT"  FOR  OUTPUT  AS  #2 
560  FOR  1-1  TO  2377 

570  PRINT  #2,  CAND(I) ;X1A(I) ;X2A(I) ;X3A(I) ;X4A(I) 

580  NEXT  I 
590  CLOSE  #2 

600  INPUT  "PUT  2ND  OUTPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

610  OPEN  "B: LITTLEX2.DAT"  FOR  OUTPUT  AS  #2 
620  FOR  1-2378  TO  MAX 

630  PRINT  #2,  CAND(I) ;X1A(I) ;X2A(I) ;X3A(I) ;X4A(I) 

640  NEXT  I 
650  CLOSE  #2 

660  REM  ********************************************************* 

670  REM  THIS  MODULE  PERFORMS  TERMINATION  HOUSEKEEPING 

680  REM  ********************************************************* 

690  LPRINT  DATE?;"  ";TIME$ 

700  FOR  1-1  TO  4 

710  LPRINT  "XMIN(" ;!;■)-  ";XMIN(I); 

720  LPRINT  "  XMAX(" ; I ; ")-  ";XMAX(I) 

730  NEXT  I 
740  END 

7  50  REM  ********************************************************* 

760  REM  THIS  MODULE  COMPUTES  RAW  CRITERION  SCORES 

770  REM  ********************************************************* 

780  REM  READ  IN  PARAMETER  VALUES 

790  FOR  I— FIRST  TO  LAST 

800  IF  1-1  THEN  ZIP$-INPUT$(7 ,  #1) 

810  IF  1-1  THEN  CAND(I)— 0 

820  IF  I>1  THEN  INPUT  #1,  CAND(I) 

830  FOR  J-l  TO  41 
840  INPUT  #1,  Y(J) 

850  NEXT  J 

860  REM  COMPUTE  XI 

870  Z13-Y(24) -Y(25) 

880  Z1222-(Y(20)*Y(21)*. 001)+(Y(22)*Y(17) )+(Y(23)*Y(18) ) 
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890  Z1221-(Y(11)*Y(12))/Y(19) 

900  Z122-Z1221*Z1222* . 001 

910  Z1212— Y(14)+(Y(15)*Y(17) )+(Y(16)*Y(18)) 

920  Z1211— (Y(11)*Y(12) )/Y(13) 

930  Z121-Z1211*Z1212* . 001 

940  Z12-Z121+Z122 

950  Z1121B— (Y(3)/Y(4) ) a(1/Y(4) ) 

960  Z1121F-(Y(5)/Y(6) ) A (1/Y(6) ) 

970  Z1121M-(Y(7)/Y(8) )  £1/Y(8)) 

980  Z1121C-(Y(9)A(10))  UAO-0)) 

990  Z1121-Z1121B*Z1121F*Z1121M*Z1121C 

1000  Z112-l/(((Y(2)+2)/4)*Z1121) 

1010  Zlll-Y(l) 

1020  Z11-Z111*Z112 

1030  X1A(I)-1/(Z11+Z12+Z13) 

1040  IF  X1A(I)<XMIN(1)  THEN  XMIN(1)-X1A(I) 

1050  IF  XlA(I)>XMAX(l)  THEN  XMAX(1)-X1A(I) 

1060  REM  COMPUTE  X2 
1070  Z23-Y(31)A(32) 

1080  Z22-Y(29)*Y£30) 

1090  Z21-((Y(26)  3)*Y(27))A(28) 

1100  X2A(I)-1/(Z21*Z22*Z23) 

1110  IF  X2A(I)<XMIN(2)  THEN  XMIN(2)-X2A(I) 

1120  IF  X2A(I)>XMAX(2)  THEN  XMAX(2)-X2A(I) 

1130  REM  COMPUTE  X3 
1140  Z33-0 

1150  A-(Y(37)*Y(35))/Y(13) 

1160  D— 1 

1170  FOR  JT0  TO  Y(37) -Y(38) 

1180  B-A  J 

1190  C-EXP(-A) 

1200  IF  J>1  THEN  D-D*J 

1210  Z33-Z33+((B*C)/D) 

1220  NEXT  J 

1230  Z3222-l/( (1/Y(13) )+(l/Y(19) ) ) 

1240  Z32212-Y(22)+Y(23) 

1250  Z32211-Y(15)+Y(16) 

1260  Z3221-((Z32211/Y(13))+(Z32212/Y(19)))/((lA(13))  +  (l/YO-9))) 

1270  Z322-(Y(35)*Z3221)/Z3222 

1280  Z321-Y(35)+(Y(36)*30*24) 

1290  Z32-(Z321-Z322)/Z321 

1300  Z31-(Y(33)+(Y(34)/52))*Z112 

1310  Z31-Z31+1 

1320  X3A(I)-(Z32*Z33)/Z31 

1330  IF  X3A(I)<XMIN(3)  THEN  XMIN(3)-X3A(I) 

1340  IF  X3A(I)>XMAX(3)  THEN  XMAX(3)-X3A(I) 

1350  REM  COMPUTE  X4 

1360  Z42-(Z3222*Y(39) )/(Z3221*Y(40)*Y(41) ) 

1370  Z41-Z33*Y(39)/W0) 

1380  X4A(I)-Z41*Z42 

1390  IF  X4A(I)<XMIN(4)  THEN  XMIN(4)-X4A(I) 

1400  IF  X4A(I)>XMAX(4)  THEN  XMAX(4)-X4A(I) 
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1410  IF  I  HOD  200-0  THEN  PRINT  I 
1420  NEXT  I 
1430  GOTO  460 


Appendix  C 


SOURCE  LISTING  FOR  BIG -X. BAS 


10  REM  ********************************************************** 
2  0  REM  ********************************************************** 
30  REM 

40  REM  BIG -X. BAS  CONVERTS  RAW  CRITERION  SCORES  TO  COMPARABLE 
50  REM  UNITS,  DENOTED  AS  BIG  X'S.  IT  OBTAINS  THE  RAW  SCORES 
60  REM  FROM  B : LITTLEX1 . DAT  AND  BrLITTLEX2.DAT.  RESULTS  ARE 
70  REM  STORED  IN  B : BIGX1.DAT  AND  B: BIGX2.DAT. 

80  REM 

90  REM  ********************************************************** 
100  REM  ********************************************************* 
110  REM  THIS  MODULE  PERFORMS  INITIAL  HOUSEKEEPING 
120  REM  ********************************************************* 
130  PRINT  TIME$ 

140  OPTION  BASE  1 
150  FORM$-"# . ####" 

160  DEFINT  F, I , J ,K 
170  MAX-4753 
180  DIM  CAND(MAX) 

190  DIM  XlA(MAX) 

200  DIM  X2A(MAX) 

210  DIM  X3A(MAX) 

220  DIM  X4A(MAX) 

230  DIM  BIGX(15) 

240  DIM  XMIN(15) 

250  DIM  XMAX(15) 

260  FOR  1-1  TO  15 
270  XMIN-9999 

280  XMAX-0 
290  NEXT  I 

300  REM  ********************************************************* 
310  REM  THIS  MODULE  READS  IN  RAW  CRITERION  SCORES 
320  REM  ********************************************************* 
330  INPUT  "PUT  1ST  INPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

340  OPEN  "B.-LITTLEX1.DAT"  FOR  INPUT  AS  #1 
350  FOR  1-1  TO  2377 

360  INPUT  #1,  CAND(I) ,X1A(I) ,X2A(I) ,X3A(I) ,X4A(I) 

370  NEXT  I 
380  CLOSE  #1 

390  INPUT  "PUT  2ND  INPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

400  OPEN  "B: LITTLEX2.DAT"  FOR  INPUT  AS  #1 
410  FOR  1-2378  TO  MAX 
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420  INPUT  #1,  CAND(I) ,X1A(I) ,X2A(I) ,X3A(I) ,X4A(I) 

430  NEXT  I 
440  CLOSE  #1 

460  REM  THIS  MODULE  COMPUTES  BIG  X'S  AND  INTERSECTION  X'S 

470  REM  ********************************************************* 

480  INPUT  "PUT  1ST  OUTPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

490  OPEN  "B: BIGX1.DAT"  FOR  OUTPUT  AS  #2 

500  FOR  1-1  TO  MAX 

510  IF  102378  THEN  GOTO  550 

520  CLOSE  #2 

530  INPUT  "PUT  2ND  OUTPUT  DISK  IN  B:  AND  PRESS  ENTER" ;NIL$ 

540  OPEN  "B: BIGX2.DAT"  FOR  OUTPUT  AS  #2 

550  PRINT  #2,  CAND(I) ; 

560  Xl-0 
570  X2-0 

580  X3-0 

590  X4-0 

600  XI 2-0 

610  XI 3-0 

620  X14-0 

630  X23-0 

640  X24-0 

650  X34-0 

660  X123-0 

670  XI 24-0 

680  XI 34-0 
690  X234-0 

700  X1234-0 

710  REM  OBTAIN  COUNTS  FOR  MARGINAL  X'S 
720  FOR  J-l  TO  MAX 

730  IF  X1A(I)>— XlA(J)  THEN  Xl-Xl+1 

740  IF  X2A(I)>— X2A(J)  THEN  X2-X2+1 

750  IF  X3A(I)>— X3A(J)  THEN  X3-X3+1 

760  IF  X4A(I)>— X4A(J)  THEN  X4-X4+1 

770  REM  OBTAIN  COUNTS  FOR  INTERSECTION  X'S 

780  IF  X1A(I)>— X1A(J)  AND  X2A(I)>-X2A(J)  THEN  X12-X12+1 

790  IF  X1A(I)>— XlA(J)  AND  X3A(I)>-X3A(J)  THEN  X13-X13+1 

800  IF  X1A ( I ) >— XlA ( J )  AND  X4A(I)>-X4A(J)  THEN  X14-X14+1 

810  IF  X2A(I)>— X2A(J)  AND  X3A(I)>-X3A(J)  THEN  X23-X23+1 

820  IF  X2A(I)>-X2A(J)  AND  X4A(I)>-X4A(J)  THEN  X24-X24+1 

830  IF  X3A(I)>— X3A(J)  AND  X4A(I)>-X4A(J)  THEN  X34-X34+1 

840  IF  X1A(I)>— XlA(J)  AND  X2A(I)>-X2A(J)  AND  X3A(I)>-X3A(J)  THEN 

X123-X123+1 

850  IF  X1A(I)>— X1A(J)  AND  X2A(I)>-X2A(J)  AND  X4A(I)>— X4A( J)  THEN 

X124-X124+1 

860  IF  X1A(I)>— XlA(J)  AND  X3A(I )>-X3A(J)  AND  X4A(I)>-X4A(J)  THEN 

X134-X134+1 

870  IF  X2A(I)>— X2A(J)  AND  X3A(I)>-X3A(J)  AND  X4A(I)>-X4A(J)  THEN 

X234-X234+1 

880  IF  X1A(I)>— XlA(J)  AND  X2A(I)>-X2A(J)  AND  X3A(I)>-X3A(J)  AND 

X4A(I)>— X4A(J)  THEN  X1234-X1234+1 
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890  NEXT  J 

900  REM  CONVERT  COUNTS  TO  BIG  X  VALUES 

910  BIGX(l)— Xl/MAX 

920  BIGX(2)-X2/MAX 

930  BIGX(3)-X3/MAX 

940  BIGX  (  4  )  -X4  /MAX 

950  BIGX(5)-X12/MAX 

960  BIGX (6)— XI 3 /MAX 

970  BIGX(7)— X14/MAX 

980  BIGX(8)— X23/MAX 

990  BIGX(9)— X24/MAX 

1000  BIGX(IO)— X34/MAX 

1010  BIGX ( 11 )-X12 3/MAX 

1020  BIGX(12)-X124/MAX 

1030  BIGX(13)-X134/MAX 

1040  B IGX( 14 )-X2 34/MAX 

1050  BIGX(15)«X1234/MAX 

1060  FOR  J-l  TO  15 

1070  IF  BIGX(J)<XMIN(J)  THEN  XMIN(J)-BIGX(J) 

1080  IF  BIGX(J)>XMAX(J)  THEN  XMAX(J)-BIGX(J) 

1090  PRINT  #2,  USING  FORM$ ;  BIGX(J) ; 

1100  NEXT  J 

1110  PRINT  #2,  "  " 

1120  PRINT  I;"  ";TIME$ 

1130  NEXT  I 

1140  REM  ******************************************************** 
1150  REM  THIS  MODULE  PERFORMS  TERMINATION  HOUSEKEEPING 
1160  REM  ******************************************************** 
1170  CLOSE  #2 

1180  LPRINT  DATE$ ; "  ";TIME$ 

1190  FOR  1-1  TO  15 

1200  LPRINT  "XMIN(" ; I ; ";XMIN(I); 

1210  LPRINT  "  XMAX(" ;I; ")-  ";XMAX(I) 

1220  NEXT  I 
1230  END 


Appendix  D 

SOURCE  LISTING  FOR  CF.BAS 


30  REM 

40  REM  CF.BAS  COMPUTES  CRITERION  FUNCTION  VALUES  FOR 
50  REM  CANDIDATE  SYSTEMS  BASED  ON  BIG  X  VALUES  STORED 
60  REM  IN  TWO  DATA  FILES.  THE  RESULTS  ARE  STOkED  IN 
70  REM  CF-OUT.DAT. 

80  REM 

110  REM  THIS  MODULE  PERFORMS  INITIAL  HOUSEKEEPING 

L^LK)  K  KM  XXXTXXXX^^XXTTXXXXXAXXXRRVfXXXR  «XX«WXXXXXXXXTXXXXARnR7r>fXXX 

130  PRINT  TIME$ 

140  OPTION  BASE  1 
150  FORM$— "# . ####" 

160  DEFINT  F.I.J.K 
170  MAX-4753 
180  DIM  BIGX(15) 

190  DIM  A(15) 

200  DIM  BIGA(15) 

210  CFMIN-9999 
220  CFMAX-0 

240  REM  THIS  MODULE  COMPUTES  RELATIVE  WEIGHTS 

260  OPEN  "BIGX1.DAT"  FOR  INPUT  AS  #1 

270  OPEN  "B: CF-OUT.DAT"  FOR  OUTPUT  AS  #2 

280  FOR  1-1  TO  MAX 

290  IF  102378  THEN  GOTO  320 

300  CLOSE  #1 

310  OPEN  "BIGX2.DAT"  FOR  INPUT  AS  #1 
320  INPUT  #1,  CAND 
330  FOR  J-l  TO  15 


340  INPUT  #1,  BIGX(J) 

350  NEXT  J 

360  PRINT  #2,  CAND; 

370  BIGASUM— 0 

380  FOR  J-l  TO  15 
390  REM  COMPUTE  A'S 

400  IF  BIGX(J)<-. 01  THEN  A(l)-1  ELSE  A(l)- 
ABS(L0G10(BIGX(J) ) )/2 
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410  IF  BIGX( J)<— . 5  THEN  A(2)-.5  gLSE  A(2)-BIGX(J) 

420  A(3)— (2*BIGX(J) ) - (2*(BIGX(J)  2)) 

430  IF  BIGX(J)<— . 5  THEN  A(4)-1-BIGX(J)  ELSE  A(4)-.5 

440  A(5)-0 

450  IF  BIGX(J)<— . 5  THEN  A(6)-.3  ELSE  A(6)-. 6- ( (3/5)*BIGX(J) ) 
460  A(7)-0 

470  A(8)-.7 

480  IF  BIGX(J)<-.5  THEN  A(9)-1-BIGX(J)  ELSE  A(9)-BIGX(J) 

490  A(10)-0 

500  FOR  K-ll  TO  15 

510  A(K)-0 

520  NEXT  K 

530  REM  NORMALIZE  A'S  VERTICALLY 
540  ASUM-0 

550  FOR  K-l  TO  15 

560  ASUM— ASUM+A  (  K ) 

570  NEXT  K 

580  BIGA ( J ) — A ( J ) /ASUM 

590  BIGASUM-BIGASUM+BIGA(J) 

600  NEXT  J 

610  REM  NORMALIZE  A'S  HORIZONTALLY 
620  FOR  J-l  TO  15 
630  BIGA( J )— BIGA( J ) /BIGASUM 

640  NEXT  J 

650  REM  MODIFY  THE  A'S  TO  INCLUDE  ALL  HIGHER-ORDER  INTERSECTIONS 
660  BIGA(l)— BIGA(1)+BIGA(5)+BIGA(6)+BIGA(7)+BIGA(11)+BIGA(12)+ 
BIGA(13)+BIGA<15) 

670  BIGA(2)-BIGA(2)+BIGA(5)+BIGA(8)+BIGA(9)+BIGA(11)+BIGA(12)+ 

BIGA(14)+BIGA(15) 

680  BIGA(3)-BIGA(3)+BIGA(6)+BIGA(8)+BIGA(10)+BIGA(11)+BIGA(13)+ 

BIGA(14)+BIGA(15) 

690  BIGA(4)-BIGA(4)+BIGA(7)+BIGA(9)+BIGA(10)+BIGA(12)+BIGA(13)+ 

BIGA(14)+BIGA(15) 

700  BIGA(5)-BIGA(5)+BIGA(11)+BIGA(12)+BIGA(15) 

710  BIGA(6)-BIGA(6)+BIGA(11)+BIGA(13)+BIGA(15) 

720  BIGA(7)-BIGA(7)+BIGA(12)+BIGA(13)+BIGA(15) 

730  BIGA(8)-BIGA(8)+BIGA(11)+BIGA(14)+BIGA(15) 

740  BIGA(9)-BIGA(9)+BIGA(12)+BIGA(14)+BIGA(15) 

750  BIGA(10)-BIGA(10)+BIGA(13)+BIGA(14)+BIGA(15) 

760  BIGA(ll)— BIGA(11)+BIGA(15) 

770  BIGA(12)-BIGA(12)+BIGA(15) 

780  BIGA(13)-BIGA(13)+BIGA(15) 

790  BIGA(14)-BIGA(14)+BIGA(15) 

810  REM  THIS  ROUTINE  COMPUTES  CF 

820  REM  ********************************************************* 

830  CF-0 

840  FOR  J-l  TO  4 

850  CF-CF+(BIGA(J)*BIGX(J) ) 

860  NEXT  J 

870  FOR  J-5  TO  10 

880  CF-CF- (BIGA(J)*B1GX(J) ) 
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890  NEXT  J 

900  FOR  J-ll  TO  14 

910  CF— CF+(BIGA(J)*BIGX(J) ) 

920  NEXT  J 

930  CF-CF- (BIGA(15)*BIGX(15) ) 

940  PRINT  #2,  USING  FORM$ ;  CF 

950  IF  CFCCFMIN  THEN  WORST-CAND 

960  IF  CFCCFMIN  THEN  CFMIN-CF 

970  IF  CF>CFMAX  THEN  BEST-CAND 

980  IF  CF>CFMAX  THEN  CFMAX-CF 

990  PRINT  CAND;CF; WORST ;CFMIN; BEST ;CFMAX;TIME$ 

1000  NEXT  I 

1010  CLOSE  #1 

1020  CLOSE  #2 

1030  END 
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